hadoop3.1.2伪集群安装

hadoop伪集群安装

默认已经从网上下载并解压安装好jdk1.8和hadoop3.1.2，配置好环境变量

修改hadoop目录下的etc/hadoop中的5个文件

1. core-site.xml

	<configuration>
		  <property>
			    <name>fs.defaultFS</name>
			    <value>hdfs://worker0:9000</value>
		  </property>
		  <property>
			    <name>hadoop.tmp.dir</name>
			    <value>/usr/local/hadoop-3.1.2/data/</value>
	 	 </property>
	</configuration>

2.hdfs-site.xml

	<configuration>
		<property>
			<name>dfs.replication</name>
			<value>1</value>
		</property>
	</configuration>

3.mapred-site.xml

	<configuration>
		<property>
			<name>mapreduce.framework.name</name>
			<value>yarn</value>
		</property>
	</configuration>

4.yarn-site.xml

	<configuration>
		<property>
			<name>yarn.resourcemanager.hostname</name>
			<value>worker0</value>
		</property>
		<property>
			<name>yarn.nodemanager.aux-services</name>
			<value>mapreduce_shuffle</value>
		</property>
	</configuration>

5. hadoop-env.sh

	export JAVA_HOME=/usr/local/jdk1.8.0_211
	export HDFS_NAMENODE_USER=root
	export HDFS_DATANODE_USER=root