su - hdfs

PI Estimator Testing:
hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 10 100

TeraGen/TeraSort/TeraValidate Testing:
1. hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar teragen 10485760000 /user/hduser/input  # 10,485,760,000=1000GB=1TB

2. hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar terasort /user/hduser/input /user/hduser/output

3. hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar teravalidate /user/hduser/output /user/hduser/validate

TestDFSIO Testing:
Usage:
hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar TestDFSIO -read | -write | -clean [-nrFiles N] [-fileSize MB]

1. hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar TestDFSIO -write -nrFiles 10 -fileSize 1000

2. hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar TestDFSIO -read -nrFiles 10 -fileSize 1000

3. hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar TestDFSIO -clean

NameNode benchmark (nnbench):
The benchmark can simulate requests for creating, reading, renaming and deleting files on HDFS.
hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar nnbench -operation create_write -maps 12 -reduces 6 -startTime 10 -blockSize 1 -bytesToWrite 0 -numberOfFiles 1000 -replicationFactorPerFile 3 -readFileAfterOpen true -baseDir /benchmarks/NNBench-`hostname -s`

MapReduce benchmark (mrbench):
hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar mrbench -numRuns 50



TestDFSIO, nnbench, mrbench
hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.2.0-tests.jar

TeraGen/TeraSort/TeraValidate , PI Estimator
hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar


Running SparkPi in YARN Cluster/Client Mode
( you should first install spark service with cloudera manager )
spark-submit --class org.apache.spark.examples.SparkPi --deploy-mode cluster --master yarn /opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark/examples/lib/spark-examples-1.1.0-cdh5.2.0-hadoop2.5.0-cdh5.2.0.jar 10


spark-submit --class org.apache.spark.examples.SparkPi --deploy-mode client --master yarn /opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark/examples/lib/spark-examples-1.1.0-cdh5.2.0-hadoop2.5.0-cdh5.2.0.jar 10