hadoop完全分布式

hadoop搭建完全分布式集群

配置集群
（1）核心配置文件
配置core-site.xml
[atguigu@hadoop102 ~]$ cd $HADOO{P}_{H}OME/etc/hadoop[atguigu@hadoop102hadoop]$ vim core-site.xml
文件内容如下：

<?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> fs.defaultFS hdfs://hadoop102:9820 hadoop.tmp.dir /opt/module/hadoop-3.1.3/data

<property>
    <name>hadoop.http.staticuser.user</name>
    <value>atguigu</value>

<property>
    <name>hadoop.proxyuser.atguigu.hosts</name>
    <value>*</value>

hadoop.proxyuser.atguigu.groups * hadoop.proxyuser.atguigu.groups * （2）HDFS配置文件 配置hdfs-site.xml [atguigu@hadoop102 hadoop]$ vim hdfs-site.xml 文件内容如下： <?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> dfs.namenode.http-address hadoop102:9870 dfs.namenode.secondary.http-address hadoop104:9868 （3）YARN配置文件 配置yarn-site.xml [atguigu@hadoop102 hadoop]$ vim yarn-site.xml 文件内容如下： <?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> yarn.nodemanager.aux-services mapreduce_shuffle yarn.resourcemanager.hostname hadoop103 yarn.nodemanager.env-whitelist JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME yarn.scheduler.minimum-allocation-mb 512 yarn.scheduler.maximum-allocation-mb 4096 yarn.nodemanager.resource.memory-mb 4096 yarn.nodemanager.pmem-check-enabled false yarn.nodemanager.vmem-check-enabled false （4）MapReduce配置文件 配置mapred-site.xml [atguigu@hadoop102 hadoop]$ vim mapred-site.xml 文件内容如下： <?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> mapreduce.framework.name yarn