1.core-site.xml
<property>
<name>fs.defaultFS</name>
<value>hdfs://192.168.0.111:9999</value>
</property>
hadoop访问地址
2.hdfs-site.xml
<property>
<name>dfs.replication</name>
<value>3</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>/home/hadoop/bigdata/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>/home/hadoop/bigdata/data</value>
</property>
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
// centeOS 不配这个导致50070无法访问
<property>
<name>dfs.http.address</name>
<value>0.0.0.0:50070</value>
</property>
3.yarn-site.xml
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>node1:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>node1:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>node1:8031</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>node1:8033</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>node1:8088</value>
</property>
<property>
<name>yarn.nodemanager.pmem-check-enabled</name>
<value>false</value>
</property>
<property>
<name>yarn.nodemanager.vmem-check-enabled</name>
<value>false</value>
</property>
4.mapred-site.xml
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapred.job.tracker</name>
<value>node3:9001</value>
</property>
5.slave配置
node2
node3
6.配置hadoop-env.sh
export JAVA_HOME=/usr/local/jdk1.8.0_144
7.防火墙,改主机名
service iptables stop
hostname node1
8.环境变量
export HADOOP_HOME=/usr/local/hadoop-2.7.4
export JAVA_HOME=/usr/local/jdk1.8.0_144
PATH=$JAVA_HOME/bin:$PATH:$HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
9.时间同步
报错:
YarnException: Unauthorized request to start container.
This token is expired. current time is 1537779423784 found 1535101599262
Note: System times on machines may be out of sync. Check system time and time zones.
yum -y install ntp ntpdate
ntpdate cn.pool.ntp.org
10.yarn配错
2018-09-24 17:02:03,211 INFO [main] org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8030
2018-09-24 17:02:04,350 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-09-24 17:02:05,351 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
<property>
<name>yarn.resourcemanager.address</name>
<value>master:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>master:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>master:8031</value>
</property>
11.启动命令
1.启停hdfs
start-dfs.sh
stop-dfs.sh
2.启tingyarn
start-yarn.sh
stop-yarn.sh
3.启动所有
start-all.sh
stop-all.sh
启动namenode
hadoop-daemon.sh start namenode
hadoop-daemon.sh start datanode
若发现有节点未启动
注意先清空 data与name中的文件