Hadoop全分布搭建

本地模式的配置

参数文件	配置参数	参考值
hadoop-env.sh	JAVA_HOME	/root/training/jdk1.8.0_144

伪分布模式的配置

参数文件	配置参数	参考值
hadoop-env.sh	JAVA_HOME	/root/training/jdk1.8.0_144
hdfs-site.xml	dfs.replication	1
	dfs.permissions	false
core-site.xml	fs.defaultFS	hdfs://<hostname>:9000
	hadoop.tmp.dir	/root/training/hadoop-2.7.3/tmp
mapred-site.xml	mapreduce.framework.name	yarn
yarn-site.xml	yarn.resourcemanager.hostname	<hostname>
	yarn.nodemanager.aux-services	mapreduce_shuffle

全分布模式的配置

参数文件	配置参数	参考值
hadoop-env.sh	JAVA_HOME	/root/training/jdk1.8.0_144
hdfs-site.xml	dfs.replication	2
	dfs.permissions	false
core-site.xml	fs.defaultFS	hdfs://<hostname>:9000
	hadoop.tmp.dir	/root/training/hadoop-2.7.3/tmp
mapred-site.xml	mapreduce.framework.name	yarn
yarn-site.xml	yarn.resourcemanager.hostname	<hostname>
	yarn.nodemanager.aux-services	mapreduce_shuffle
slaves	DataNode的地址	从节点1 从节点2

 

如果出现以下警告信息：

只需要在以下两个文件中增加下面的环境变量，即可：

1. hadoop-env.sh脚本中

export JAVA_HOME=/root/training/jdk1.8.0_144

export HADOOP_HOME=/root/training/hadoop-2.7.3

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native

export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"

 

1. yarn-env.sh脚本中

export JAVA_HOME=/root/training/jdk1.8.0_144

export HADOOP_HOME=/root/training/hadoop-2.7.3

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native

export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"