CDHHadoop单机安装

最新推荐文章于 2024-04-13 16:02:30 发布

powerx_yc

最新推荐文章于 2024-04-13 16:02:30 发布

阅读量214

点赞数

文章标签：大数据 python java

原文链接：https://my.oschina.net/u/2253438/blog/1575893

版权

2019独角兽企业重金招聘Python工程师标准>>>

1.修改java_home的文件：hadoop-env.sh,yarn-env.sh,mapreduce.env.sh
export JAVA_HOME=/usr/local/devtools/jdk/jdk1.7.0_45

2.修改core-site.xml

<configuration>

    <!-- hdfs NameNode地址 -->
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://xupan001:8020</value>
    </property>

    <property>
            <name>hadoop.tmp.dir</name>
            <value>/usr/local/devtools/cdhbigdata/cdhhadoop/hadoop-2.5.0-cdh5.3.6/data/tmp</value>
    </property>

   <property>
        <name>fs.trash.interval</name>
        <value>10080</value>
  </property>


</configuration>


3.修改hdfs-site.xlm
<configuration>

    <property>
        <name>dfs.namenode.secondary.http-address</name>
        <value>xupan001:50090</value>
    </property>

    <property>
        <name>dfs.namenode.http-address</name>
        <value>xupan001:50070</value>
    </property>

    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>

    <property>
        <name>dfs.permissions</name>
        <value>false</value>
    </property>

</configuration>

4.修改yarn-site.xml
<configuration>

  <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
  </property>

  <property>
        <name>yarn.resourcemanager.hostname</name>
        <value>xupan001</value>
  </property>
<!--
  <property>
        <name>yarn.log.server.url</name>
        <value>http://xupan001:19888/jobhistory/job/</value>
  </property>
-->
  <property>
        <name>yarn.log-aggregation-enable</name>
        <value>true</value>
  </property>



  <property>
       <name>yarn.log-aggregation.retain-seconds</name>
       <value>604800</value>
  </property>


  <property>
        <name>yarn.nodemanager.resource.memory-mb</name>
        <value>4096</value>
  </property>

  <property>
        <name>yarn.nodemanager.resource.cpu-vcores</name>
        <value>4</value>
  </property>

</configuration>


5.修改mapred-site.xml
<configuration>

    <property>
         <name>mapreduce.framework.name</name>
         <value>yarn</value>
    </property>

    <property>
         <name>mapreduce.jobhistory.address</name>
         <value>xupan001:10020</value>
    </property>

    <property>
         <name>mapreduce.jobhistory.webapp.address</name>
         <value>xupan001:19888</value>
    </property>

</configuration>

6.修改slaves
xupan001

转载于:https://my.oschina.net/u/2253438/blog/1575893