Hadoop(一)伪分布式安装Hadoop

最新推荐文章于 2019-03-27 16:41:23 发布

t_6666

最新推荐文章于 2019-03-27 16:41:23 发布

阅读量591

点赞数

文章标签： hadoop jdk 终端 java 大数据

本文链接：https://blog.csdn.net/T_6666/article/details/52239975

版权

Hadoop(一)乌班图伪分布式安装Hadoop

1.打开终端

2.安装jdk，去官网中下载jdk（此处只作简单的介绍）-->解压-->配置环境

3.配置SSH免登录

4.配置hadoop环境

4.1官网路径，下载（这里使用的是2.6.4）http://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-2.6.4/hadoop-2.6.4.tar.gz

4.2解压hadoop-2.6.4.tar.gz，并重命名为hadoop

tar xzvf hadoop-2.4.0.tar.gz
mv hadoop-2.4.0 hadoop

4.3配置环境变量，内容如下，而后使之生效：source /etc/environment

4.4在这里提一句，单机模式无需仁和配置可以直进行测试,略之

4.5伪分布式模式配置：

4.5.1在当前用户目录下创建文件夹tmp/hadoop_tmp

4.5.2etc/hadoop下存放配置文件

4.5.2.1修改hadoop-env.sh

将export JAVA_HOME=${JAVA_HOME}改为你自己安装的jdk路径：

export JAVA_HOME=/usr/lib/java/jdk1.8.0_73

4.5.2.2修改core-site.xml

<configuration>
  <property>
    <name>fs.defaultFS</name>
    <value>hdfs://localhost:9000</value>
    <final>true</final>
  </property>

  <property>
    <name>hadoop.tmp.dir</name>
    <value>/home/judy/tmp/hadoop_tmp</value>
  </property>
</configuration>

4.5.2.3修改hdfs-site.xml:

<configuration>
 <property>
      <name>dfs.namenode.name.dir</name>
      <value>/home/judy/Desktop/Hadoop/hadoop/dfs/namenode</value>
      <final>true</final>
    </property>      
     <property>
       <name>dfs.datanode.data.dir</name>
       <value>/home/judy/Desktop/Hadoop/hadoop/dfs/datanode</value>
      <final>true</final>
    </property>
    <property>
      <name>dfs.http.address</name>
      <value>localhost:50070</value>
       <description>
         The address and the base port where the dfs namenode web ui will listen on.
If the port is 0 then the server will start on a free port.
      </description>
    </property>
    <property>
      <name>dfs.replication</name>
      <value>1</value>
    </property>
    <property>
      <name>dfs.permissions</name>
      <value>false</value>
    </property>
</configuration>

4.5.2.4修改mapred-site.xml

<configuration>
  <property>
      <name>mapreduce.framework.name</name>
      <value>yarn</value>
    </property>
    <property>
      <name>mapred.job.tracker</name>
      <value>hdfs://localhost:9001</value>
    </property>
        <property>  
          <name>mapred.system.dir</name>  
          <value>file:/home/ylf/hadoop/mapred/system</value>  
          <final>true</final>  
        </property>  
       
        <property>  
          <name>mapred.local.dir</name>  
          <value>file:/home/ylf/hadoop/mapred/local</value>  
          <final>true</final>  
        </property> 
</configuration>

4.2.2.5修改yarn-site.xml

<configuration>

<!-- Site specific YARN configuration properties --> <property>

    <name>yarn.nodemanager.aux-services</name>

    <value>mapreduce_shuffle</value>

    <description>shuffle service that needs to be set for Map Reduce to run</description>

  </property>

  <property>

    <name>yarn.resourcemanager.hostname</name>

    <value>localhost</value>

    <description>hostname of Resource Manager</description>

  </property>

</configuration>

4.2.2.6修改slaves文件