Hadoop Road Map - 03 - Hadoop Initialization

Please add your comments if any concern about this. Thanks in advance.

format Hadoop namenode & start hadoop

>$ hadoop namenode –format
>$ start-all.sh
>$ xcall.sh jps

Trouble shooting

Root cause: your java bin folder is not default.
Solution:

>$ sudo ln -s -f /hadoop/jdk1.8.0_211/bin/jps /usr/bin/jps

Hadoop hdfs system testing
http://s01:50070

Connect with Hadoop with Eclipse

Download the software below:
hadoop-eclipse-plugin-2.6.0.jar
jdk-8u211-linux-x64.tar.gz
eclipse-jee-2019-03-R-linux-gtk-x86_64.tar.gz

Confirm Hadoop works fine

>$ cd /hadoop/hadoop/share/hadoop/mapreduce
>$ hadoop fs -mkdir /home
>$ hadoop fs -mkdir /home/input
>$ hadoop fs -mkdir /home/output
>$ hadoop fs -put /hadoop/hadoop/LICENSE.txt /home/input
>$ hadoop jar hadoop-mapreduce-examples-3.1.2.jar wordcount /home/input/LICENSE.txt /home/output/example_count

You will see the files below while it succeeds.

Unzip & config eclipse
put hadoop-eclipse-plugin-2.6.0.jar under folder eclipse/plugins
start eclipse
Windows —> Preferences -> Hadoop Map/Reduce input Hadoop folder
Windows —> Open Perspective —> Other -> Map/Reduce
Right click Hadoop -> new Hadoop location

Input specific informations

Verification the file shown under path DFSLocations -> Hadoop