Linux 安装 Hadoop 教程

去官网下载Hadoop压缩包

Apache Hadoop

 

 

用XFTP把下载的压缩包放在opt目录下的install文件夹里

开始解压解压到opt目录下的soft

 改名为hadoop313

 改权限

 配环境变量

重新加载下

 在hadoop313目录下新建data目录

配置 vim ./core-site.xml

<configuration>
  <property>
    <name>fs.defaultFS</name>
    <value>hdfs://kb129:9000</value>
  </property>
  <property>
    <name>hadoop.tmp.dir</name>
    <value>/opt/soft/hadoop313/data</value>
  </property>
  <property>
    <name>hadoop.http.staticuser.user</name>
    <value>root</value>
  </property>
  <property>
    <name>io.file.buffer.size</name>
    <value>131073</value>
  </property>
  <property>
    <name>hadoop.proxyuser.root.hosts</name>
    <value>*</value>
  </property>
  <property>
    <name>hadoop.proxyuser.root.groups</name>
    <value>*</value>
  </property>
</configuration>

 配置 vim ./hadoop-env.sh  这里的路径是JDK的路径，可以参考我配JDK的博客

配置JDK

 配置  vim ./hdfs-site.xml

<configuration>
  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>
  <property>
    <name>dfs.namenode.name.dir</name>
    <value>/opt/soft/hadoop313/data/dfs/name</value>
  </property>
  <property>
    <name>dfs.datanode.data.dir</name>
    <value>/opt/soft/hadoop313/data/dfs/data</value>
  </property>
  <property>
    <name>dfs.permissions.enabled</name>
    <value>false</value>
  </property>
</configuration>

 配置 vim ./mapred-site.xml 

<configuration>
  <property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
  </property>
  <property>
    <name>mapreduce.jobhistory.address</name>
    <value>kb129:10020</value>
  </property>
  <property>
    <name>mapreduce.jobhistory.webapp.address</name>
    <value>kb129:19888</value>
  </property>
  <property>
    <name>mapreduce.map.memory.mb</name>
    <value>4096</value>
  </property>
  <property>
    <name>mapreduce.reduce.memory.mb</name>
    <value>4096</value>
  </property>
  <property>
    <name>mapreduce.application.classpath</name>
    <value>/opt/soft/hadoop313/etc/hadoop:/opt/soft/hadoop313/share/hadoop/common/lib/*:/opt/soft/hadoop313/share/hadoop/common/*:/opt/soft/hadoop313/share/hadoop/hdfs/*:/opt/soft/hadoop313/share/hadoop/hdfs/lib/*:/opt/soft/hadoop313/share/hadoop/mapreduce/*:/opt/soft/hadoop313/share/hadoop/mapreduce/lib/*:/opt/soft/hadoop313/share/hadoop/yarn/*:/opt/soft/hadoop313/share/hadoop/yarn/lib/*</value>
  </property>
</configuration>

配置 vim ./yarn-site.xml

<configuration>

<!-- Site specific YARN configuration properties -->
  <property>
    <name>yarn.resourcemanager.connect.retry-interval.ms</name>
    <value>20000</value>
  </property>
  <property>
    <name>yarn.resourcemanager.scheduler.class</name>  
    <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>
  </property>
  <property>
    <name>yarn.nodemanager.localizer.address</name>
    <value>kb129:8040</value>
  </property>
  <property>
    <name>yarn.nodemanager.address</name>
    <value>kb129:8050</value>
  </property>
  <property>
    <name>yarn.nodemanager.webapp.address</name>
    <value>kb129:8042</value>
  </property>
  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>
  <property>
    <name>yarn.nodemanager.local-dirs</name>
    <value>/opt/soft/hadoop313/yarndata/yarn</value>
  </property>
  <property>
    <name>yarn.nodemanager.log-dirs</name>
    <value>/opt/soft/hadoop313/yarndata/log</value>
  </property>
  <property>
    <name>yarn.nodemanager.vmem-check-enabled</name>
    <value>false</value>
  </property>
</configuration>

修改 vim ./workers

 

 

进入bin路径

 初始化（在bin路径下进行）

 出现successfully就成功了（到这一步就基本完成了）

 免密登录（这一步和Hadoop没关系，不配也行）

需要配两台虚拟机的ip地址，后面跟机器名，不然后面ssh -p 22 root@kb128会报错

 

 复制kb129的免密登录给kb128，然后就可以免密登录了

 下面就是hadoop的开启和关闭

 开启后去浏览器输入 192.168.198.129：9870 会跳转到网页，就说明成功了

记得关闭