集群问题

dataNode 连接ha两个主节点
nameNode 连接zookeeper
报错

org.apache.hadoop.ipc.Client: Retrying connect to server: THadoop7/26:8485. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSl
eep(maxRetries=10, sleepTime=1000 MILLISECONDS)

ip和端口都是通的，防火墙也已经关闭

在关闭集群时
no proxyserver to stop
和下面网址遇到的问题不一样
http://www.aboutyun.com/thread-11610-1-1.html

hive metastore
http://blog.csdn.net/skywalker_only/article/details/26219619
https://cwiki.apache.org/confluence/display/Hive/AdminManual+MetastoreAdmin
http://tangjunliang.iteye.com/blog/2037935

hadoop安全模式
http://www.cloudera.com/documentation/archive/cdh/4-x/4-2-0/CDH4-Security-Guide/cdh4sg_topic_3_9.html
http://f.dataguru.cn/thread-84564-1-1.html
https://discuss.zendesk.com/hc/en-us/articles/200933026-HDFS-goes-into-readonly-mode-and-errors-out-with-Name-node-is-in-safe-mode-

集群两个节点全都是standby ，原因是没有启动zookeeper

java.io.IOException: Broken pipe
java.io.EOFException: Premature EOF: no length prefix available

集群宕机了

YARN模式下

Unknown container. Container either has not started or has already completed or doesn't belong to this node at all.

Yarn application has already ended! It might have been killed or unable to launch application master.

Unauthorized request to start container.

这三个问题，原因是节点之间的时间和时区不一致导致的，统一集群中所有节点的时间即可，但是提交任务到spark集群，没有这样的问题

在没有开启高可用以及使用zookeeper的时候，第一次启动集群

cd ~/workspace/hadoop-2.6.0     #进入hadoop目录
hdfs namenode -format    		#格式化namenode
sbin/start-dfs.sh               #启动dfs 
sbin/start-yarn.sh              #启动yarn

集群节点的copy 数量，最低可以设置为1

如果，在yarn-site.xml 中开启了YarnShuffle，即将spark 程序放在yarn上执行，需要在/home/vpetest.cripac/softwares/hadoop-2.7.4/share/hadoop/yarn添加spark-2.1.0-yarn-shuffle.jar，这样可以解决

java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.spark.network.yarn.YarnShuffleService not found

Exception in createBlockOutputStream java.net.NoRouteToHostException: No route to host

启动集群时，yarn报这样的错，解决办法，关闭防火墙

Problem connecting to server

需要关闭防火墙

集群启动时需要保证date是一致的，同时防火墙是关闭的

Name node is in safe mode

只要在Hadoop的目录下输入：
bin/hadoop dfsadmin -safemode leave

java.net.NoRouteToHostException: 没有到主机的路由
原因是集群节点没有关闭防火墙

org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start container

同步一下的hadoop集群上的时间 ntpdate -u time.nist.gov