Hadoop-2.6.0-cdh5.7.0 部署

下载Hadoop和JDK

下载Hadoop地址：http://archive.cloudera.com/cdh5/cdh/5/hadoop-2.6.0-cdh5.7.0.tar.gz

下载jdk：推荐1.8版本

[xkhadoop@hadoop software]$ weget http://archive.cloudera.com/cdh5/cdh/5/hadoop-2.6.0-cdh5.7.0.tar.gz

解压tar包

[xkhadoop@hadoop software]$ weget http://archive.cloudera.com/cdh5/cdh/5/hadoop-2.6.0-cdh5.7.0.tar.gz

安装Hadoop

配置环境

[xkhadoop@hadoop ~]$ vi .bash_profile 
export HADOOP_HOME=/home/xkhadoop/app/hadoop-2.6.0-cdh5.7.0
export PATH=$HADOOP_HOME/bin:$PATH

修改配置文件

[xkhadoop@hadoop hadoop]$ pwd
/home/xkhadoop/app/hadoop-2.6.0-cdh5.7.0/etc/hadoop
[xkhadoop@hadoop hadoop]$ vi hadoop-env.sh 

export JAVA_HOME=/home/xkhadoop/app/jdk1.8.0_45

[xkhadoop@hadoop hadoop]$ vi core-site.xml 

<configuration>
        <property>
                <name>fs.defaultFS</name>
                <value>hdfs://xkhadoop:8020</value>
        </property>
        <property>
                <name>hadoop.tmp.dir</name>
                <value>/home/xkhadoop/app/tmp</value>
        </property>
</configuration>

[xkhadoop@hadoop hadoop]$ vi yarn-site.xml 

<configuration>
        <property>
                <name>yarn.nodemanager.aux-services</name>
                <value>mapreduce_shuffle</value>
        </property>
</configuration>

[xkhadoop@hadoop hadoop]$ vi slaves 

xkhadoop

[xkhadoop@hadoop hadoop]$ vi hdfs-site.xml

 <configuration>
        <property>
                <name>dfs.replication</name>
                <value>1</value>
        </property>
</configuration>

[xkhadoop@hadoop hadoop]$ cp mapred-site.xml.template mapred-site.xml
[xkhadoop@hadoop hadoop]$ vi mapred-site.xml

<configuration>
        <property>
                <name>mapreduce.framework.name</name>
                <value>yarn</value>
        </property>
</configuration>

重置namenode

[xkhadoop@hadoop hadoop-2.6.0-cdh5.7.0]$ hdfs namenode -format

19/03/27 20:15:19 INFO namenode.FSImage: Allocated new BlockPoolId: BP-880409199-192.168.40.133-1553688919510
19/03/27 20:15:19 INFO common.Storage: Storage directory /home/xkhadoop/app/tmp/dfs/name has been successfully formatted.
19/03/27 20:15:19 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
19/03/27 20:15:19 INFO util.ExitUtil: Exiting with status 0
19/03/27 20:15:19 INFO namenode.NameNode: SHUTDOWN_MSG: 

启动Hadoop

[xkhadoop@xkhadoop sbin]$ ./start-all.sh 
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
19/03/27 20:18:41 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [xkhadoop]
The authenticity of host 'xkhadoop (192.168.40.133)' can't be established.
RSA key fingerprint is 3e:80:d8:9a:d7:4f:4e:38:d9:64:dc:2e:15:df:5a:85.
Are you sure you want to continue connecting (yes/no)? yes
xkhadoop: Warning: Permanently added 'xkhadoop,192.168.40.133' (RSA) to the list of known hosts.
xkhadoop@xkhadoop's password: 
xkhadoop: starting namenode, logging to /home/xkhadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-xkhadoop-namenode-xkhadoop.out
xkhadoop@xkhadoop's password: 
xkhadoop: starting datanode, logging to /home/xkhadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-xkhadoop-datanode-xkhadoop.out
Starting secondary namenodes [0.0.0.0]
The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established.
RSA key fingerprint is 3e:80:d8:9a:d7:4f:4e:38:d9:64:dc:2e:15:df:5a:85.
Are you sure you want to continue connecting (yes/no)? yes
0.0.0.0: Warning: Permanently added '0.0.0.0' (RSA) to the list of known hosts.
xkhadoop@0.0.0.0's password: 
0.0.0.0: starting secondarynamenode, logging to /home/xkhadoop/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-xkhadoop-secondarynamenode-xkhadoop.out
19/03/27 20:19:27 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
starting yarn daemons
starting resourcemanager, logging to /home/xkhadoop/app/hadoop-2.6.0-cdh5.7.0/logs/yarn-xkhadoop-resourcemanager-xkhadoop.out
xkhadoop@xkhadoop's password: 
xkhadoop: starting nodemanager, logging to /home/xkhadoop/app/hadoop-2.6.0-cdh5.7.0/logs/yarn-xkhadoop-nodemanager-xkhadoop.out