Hadoop 2.6 Single Node Cluster 安裝指令

1.安裝JDK

java -version sudo apt-get update sudo apt-get install default-jdk java -version update-alternatives --display java

2.設定 SSH 無密碼登入

sudo apt-get install ssh sudo apt-get install rsync ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa ll /home/hduser/.ssh ll ~/.ssh cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

3.下載安裝Hadoop

wget http://ftp.twaren.net/Unix/Web/apache/hadoop/common/hadoop-2.6.0/hadoop-2.6.0.tar.gz sudo tar -zxvf hadoop-2.6.0.tar.gz sudo mv hadoop-2.6.0 /usr/local/hadoop ll /usr/local/hadoop

4.設定Hadoop環境變數

修改~/.bashrc

sudo gedit ~/.bashrc

輸入下列內容

export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 export HADOOP_HOME=/usr/local/hadoop export PATH=$PATH:$HADOOP_HOME/bin export PATH=$PATH:$HADOOP_HOME/sbin export HADOOP_MAPRED_HOME=$HADOOP_HOME export HADOOP_COMMON_HOME=$HADOOP_HOME export HADOOP_HDFS_HOME=$HADOOP_HOME export YARN_HOME=$HADOOP_HOME export HADOOP_COMMON_HOME=$HADOOP_HOME export HADOOP_HDFS_HOME=$HADOOP_HOME export YARN_HOME=$HADOOP_HOME export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib" export JAVA_LIBRARY_PATH=$HADOOP_HOME/lib/native:$JAVA_LIBRARY_PATH

讓~/.bashrc修改生效

source ~/.bashrc

5.修改Hadoop組態設定檔

Step1   修改hadoop-env.sh

sudo gedit /usr/local/hadoop/etc/hadoop/hadoop-env.sh

輸入下列內容:

export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64

Step2 修改core-site.xml

sudo gedit /usr/local/hadoop/etc/hadoop/core-site.xml

在之間，輸入下列內容:

fs.default.name hdfs://localhost:9000  

Step3 修改yarn-site.xml

sudo gedit /usr/local/hadoop/etc/hadoop/yarn-site.xml

在之間，輸入下列內容:

yarn.nodemanager.aux-services mapreduce_shuffle yarn.nodemanager.aux-services.mapreduce.shuffle.class org.apache.hadoop.mapred.ShuffleHandler

Step4 修改mapred-site.xml

sudo cp /usr/local/hadoop/etc/hadoop/mapred-site.xml.template /usr/local/hadoop/etc/hadoop/mapred-site.xml sudo gedit /usr/local/hadoop/etc/hadoop/mapred-site.xml

在之間，輸入下列內容:

mapreduce.framework.name yarn

Step5 修改hdfs-site.xml

sudo gedit /usr/local/hadoop/etc/hadoop/hdfs-site.xml

在之間，輸入下列內容:

   dfs.replication    3    dfs.namenode.name.dir   file:/usr/local/hadoop/hadoop_data/hdfs/namenode    dfs.datanode.data.dir   file:/usr/local/hadoop/hadoop_data/hdfs/datanode  

6.建立與格式化HDFS 目錄

sudo mkdir -p /usr/local/hadoop/hadoop_data/hdfs/namenode sudo mkdir -p /usr/local/hadoop/hadoop_data/hdfs/datanode sudo chown hduser:hduser -R /usr/local/hadoop hadoop namenode -format

7.啟動Hadoop
啟動start-dfs.sh，再啟動 start-yarn.sh

start-dfs.sh start-yarn.sh

或
啟動全部

start-all.sh

查看目前所執行的行程

jps

8.開啟Hadoop Resource­Manager Web介面
Hadoop Resource­Manager Web介面網址

http://localhost:8088/

9.NameNode HDFS Web介面
開啟HDFS Web UI網址

http://localhost:50070/