2020-08-21Hadoop在Linux中的安装配置

资源 https://pan.baidu.com/s/1uPa7TAFNEJ5ZSUEaJW1VYg
密码：fqi0

1. 进入/opt 把文件拖进去
   cd /opt
   

2. 解压 tar -zxvf hadoop-2.6.0-cdh5.14.2.tar.gz
   

3. 配置环境： vi /etc/profile
   在末行输入

export PATH USER LOGNAME MAIL HOSTNAME HISTSIZE HISTCONTROL
export JAVA_HOME=/opt/jdk1.8.0_221
export CLSASSPATH=.:$JAVA_HOME/lib/rt.jar:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar
export HADOOP_HOME=/opt/hadoop
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_INSTALL=$HADOOP_HOME
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH

4. 将hadoop-2.6.0-cdh5.14.2改成 hadoop
   

5. 检查是否成功 source /etc/profile
   hadoop version
   

6. 进hadoop里面的hadoop
   

7. 进 hadoop-env.sh里面
   

8. 进入 vi core-site.xml
   

 <property>
    <name>fs.defaultFS</name>
    <value>hdfs://192.168.137.103:9000</value>
  </property>
  <property>
    <name>hadoop.tmp.dir</name>
    <value>/opt/hadoop/tmp</value>
  </property>
  <property>
   <name>hadoop.proxyuser.root.hosts</name>
   <value>*</value>
  </property>
  <property>
   <name>hadoop.proxyuser.root.groups</name>
   <value>*</value>
  </property>
  <property>
    <name>hadoop.native.lib</name>
    <value>false</value>
    <description>Should native hadoop libraries, if present, be used.
    </description>
</property>

9. 进入 vi hdfs-site.xml
   

  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>
  <property>
  <name>dfs.namenode.secondary.http-address</name>
  <value>192.168.137.103:50090</value>
  </property>

10. 进入 vi mapred-site.xml （由mapred-site.xml.template 改名而来）
    

  <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
  </property>
  <property>
        <name>mapreduce.jobhistory.address</name>
        <value>192.168.137.103:10020</value>
  </property>
  <property>
        <name>mapreduce.jobhistory.webapp.address</name>
        <value>192.168.137.103:19888</value>
  </property>

11. 进入 vi yarn-site.xml
    

  <property>
    <name>yarn.resourcemanager.hostname</name>
    <value>hadoop03</value>
  </property>
  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>
  <property>
    <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
  </property>
<!-- 日志聚集功能使用 -->
<property>
    <name>yarn.log-aggregation-enable</name>
    <value>true</value>
</property>
<!-- 日志保留时间设置7天 -->
<property>
    <name>yarn.log-aggregation.retain-seconds</name>
    <value>604800</value>
</property>

12. 进入 vi slaves
    
    13.配置互信 vi /etc/hosts
    
    输入 ssh-keygen 生成密钥
    把公钥拷贝到秘钥库中：cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

传送给想要互信的主机：scp root@主机名:~/.ssh/id_rsa.pub ~/.ssh/id_rsa.pub

输入 ssh 主机名 查看书否成功 成功则exit退出

13. 格式化 hadoop namenode -format
    
14. 开启 start-all.sh 关闭stop-all.sh
15. 检测开起 后 输入 jps 查看进程
    
16. 在网页导航栏输入ip:9000