Hadoop CLI MiniCluster

CLI MiniCluster

Use the climonicluster, users can simply start and stop a single-node hadoop cluster witha single command, and without the need to set any environment variables ormanage configuration files. The cli monicluster starts both a mapreduce and hdfclusters. This is useful cases where users want to quickly experiment with areal hadoop cluyster or test non-java programs that rely on significant hadoopfunctionality.

From inside theroot directory of the extracted tarball, you can tsrat the cli miniclusterusing the following command:

$ bin/hadoop jarhadoop-test-*.jar minicluster -jtport JT_PORT -nnport NN_PORT

In the example command above,JT_PORT and NN_PORT should be replaced by the user's choice of these portnumbers. If not specified, random free ports will be used.

$ bin/hadoop jarhadoop-test-*.jar minicluster -jtport JT_PORT -nnport NN_PORT

Unknown program'minicluster' chosen.

Valid program namesare:

  DFSCIOTest: Distributed i/o benchmark oflibhdfs.

  DistributedFSCheck: Distributed checkup ofthe file system consistency.

  MRReliabilityTest: A program that tests thereliability of the MR framework by injecting faults/failures

  TestDFSIO: Distributed i/o benchmark.

  dfsthroughput: measure hdfs throughput

  filebench: BenchmarkSequenceFile(Input|Output)Format (block,record compressed and uncompressed),Text(Input|Output)Format (compressed and uncompressed)

  loadgen: Generic map/reduce load generator

  mapredtest: A map/reduce test check.

  mrbench: A map/reduce benchmark that cancreate many small jobs

  nnbench: A benchmark that stresses thenamenode.

  testarrayfile: A test for flat files ofbinary key/value pairs.

  testbigmapoutput: A map/reduce program thatworks on a very big non-splittable file and does identity map/reduce

  testfilesystem: A test for FileSystemread/write.

  testipc: A test for ipc.

  testmapredsort: A map/reduce program thatvalidates the map-reduce framework's sort.

  testrpc: A test for rpc.

  testsequencefile: A test for flat files ofbinary key value pairs.

  testsequencefileinputformat: A test forsequence file input format.

  testsetfile: A test for flat files of binarykey/value pairs.

  testtextinputformat: A test for text inputformat.

  threadedmapbench: A map/reduce benchmark thatcompares the performance of maps with multiple spills over maps with 1 spill