hadoop NameNode 无法启动

最新推荐文章于 2022-04-04 00:30:02 发布
最新推荐文章于 2022-04-04 00:30:02 发布
阅读量1k 收藏 2
点赞数
文章标签： 大数据 java git
原文链接：https://my.oschina.net/zxdzyxzx/blog/2250765
版权 
2018-10-21 23:42:29,245 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = bigdata01/192.168.137.101
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 2.9.0
STARTUP_MSG:   classpath = /usr/local/hadoop-2.9.0/etc/hadoop:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/nimbus-jose-jwt-3.9.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/commons-configur
ation-1.6.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/commons-cli-1.2.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/commons-net-3.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/jersey-core-1.9.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/guava-11.0.2.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/gson-2.2.4.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/log4j-1.2.17.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/jettison-1.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/commons-digester-1.8.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/stax2-api-3.1.4.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/xz-1.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/hadoop-auth-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/stax-api-1.0-2.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/jetty-6.1.26.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/curator-recipes-2.7.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/commons-io-2.4.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/curator-framework-2.7.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/json-smart-1.1.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/commons-lang-2.6.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/commons-collections-3.2.2.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/jersey-server-1.9.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/jets3t-0.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/xmlenc-0.52.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/hamcrest-core-1.3.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/servlet-api-2.5.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/commons-logging-1.1.3.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/commons-codec-1.4.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/paranamer-2.3.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/jcip-annotations-1.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/zookeeper-3.4.6.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/activation-1.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/jsch-0.1.54.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/jersey-json-1.9.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/jetty-util-6.1.26.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/commons-math3-3.1.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/junit-4.11.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/hadoop-annotations-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/httpcore-4.4.4.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/commons-lang3-3.4.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/asm-3.2.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/jetty-sslengine-6.1.26.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/avro-1.7.7.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/httpclient-4.5.2.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/jsp-api-2.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/curator-client-2.7.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/jsr305-3.0.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/netty-3.6.2.Final.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/snappy-java-1.0.5.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/hadoop-common-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/hadoop-nfs-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/common/hadoop-common-2.9.0-tests.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/okhttp-2.4.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/guava-11.0.2.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/jackson-core-2.7.8.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/jackson-annotations-2.7.8.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/commons-io-2.4.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/jackson-databind-2.7.8.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/okio-1.4.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/hadoop-hdfs-client-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/asm-3.2.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/hadoop-hdfs-client-2.9.0-tests.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/hadoop-hdfs-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/hadoop-hdfs-native-client-2.9.0-tests.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/hadoop-hdfs-client-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/hadoop-hdfs-2.9.0-tests.jar:/usr/local/hadoop-2.9.0/share/hadoop/hdfs/hadoop-hdfs-native-client-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/nimbus-jose-jwt-3.9.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/javassist-3.18.1-GA.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/java-xmlbuilder-0.4.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/commons-configuration-1.6.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/commons-cli-1.2.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/commons-net-3.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/guava-11.0.2.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/gson-2.2.4.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/woodstox-core-5.0.3.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/jettison-1.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/commons-digester-1.8.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/stax2-api-3.1.4.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/xz-1.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/curator-test-2.7.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/apacheds-i18n-2.0.0-M15.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/jetty-6.1.26.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/api-asn1-api-1.0.0-M20.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/curator-recipes-2.7.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/jersey-client-1.9.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/commons-io-2.4.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/curator-framework-2.7.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/api-util-1.0.0-M20.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/json-smart-1.1.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/commons-math-2.2.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/commons-collections-3.2.2.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/jersey-server-1.9.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/jets3t-0.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/xmlenc-0.52.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/servlet-api-2.5.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/commons-codec-1.4.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/paranamer-2.3.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/java-util-1.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/javax.inject-1.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/zookeeper-3.4.6-tests.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/jcip-annotations-1.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/activation-1.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/json-io-2.5.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/jsch-0.1.54.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/jersey-json-1.9.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/htrace-core4-4.1.0-incubating.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/guice-3.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/commons-math3-3.1.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/metrics-core-3.0.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/httpcore-4.4.4.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/fst-2.50.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/commons-lang3-3.4.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/asm-3.2.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/jetty-sslengine-6.1.26.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/commons-beanutils-1.7.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/avro-1.7.7.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/httpclient-4.5.2.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/jsp-api-2.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/curator-client-2.7.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/jsr305-3.0.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/snappy-java-1.0.5.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/hadoop-yarn-api-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/hadoop-yarn-registry-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/hadoop-yarn-server-common-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/hadoop-yarn-server-router-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/hadoop-yarn-common-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/hadoop-yarn-client-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/xz-1.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/javax.inject-1.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/guice-3.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/junit-4.11.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/asm-3.2.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/avro-1.7.7.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/snappy-java-1.0.5.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.9.0-tests.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.0.jar:/usr/local/hadoop-2.9.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.9.0.jar:/contrib/capacity-scheduler/*.jar:/contrib/capacity-scheduler/*.jar:/contrib/capacity-scheduler/*.jarSTARTUP_MSG:   build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r 756ebc8394e473ac25feac05fa493f6d612e6c50; compiled by 'arsuresh' on 2017-11-13T23:15Z
STARTUP_MSG:   java = 1.8.0_171
************************************************************/
2018-10-21 23:42:29,274 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
2018-10-21 23:42:29,278 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: createNameNode []
2018-10-21 23:42:29,705 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2018-10-21 23:42:29,902 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
2018-10-21 23:42:29,902 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system started
2018-10-21 23:42:29,959 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: fs.defaultFS is hdfs://dfsNameServices
2018-10-21 23:42:29,968 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Clients are to use dfsNameServices to access this namenode/service.
2018-10-21 23:42:30,374 INFO org.apache.hadoop.util.JvmPauseMonitor: Starting JVM pause monitor
2018-10-21 23:42:30,380 INFO org.apache.hadoop.hdfs.DFSUtil: Starting Web-server for hdfs at: http://bigdata01:50070
2018-10-21 23:42:30,532 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2018-10-21 23:42:30,555 INFO org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
2018-10-21 23:42:30,591 INFO org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.namenode is not defined
2018-10-21 23:42:30,600 INFO org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2018-10-21 23:42:30,606 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context hdfs
2018-10-21 23:42:30,607 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
2018-10-21 23:42:30,607 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
2018-10-21 23:42:30,921 INFO org.apache.hadoop.http.HttpServer2: Added filter 'org.apache.hadoop.hdfs.web.AuthFilter' (class=org.apache.hadoop.hdfs.web.AuthFilter)
2018-10-21 23:42:30,942 INFO org.apache.hadoop.http.HttpServer2: addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
2018-10-21 23:42:30,986 INFO org.apache.hadoop.http.HttpServer2: Jetty bound to port 50070
2018-10-21 23:42:30,986 INFO org.mortbay.log: jetty-6.1.26
2018-10-21 23:42:31,967 INFO org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@bigdata01:50070
2018-10-21 23:42:32,324 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Only one image storage directory (dfs.namenode.name.dir) configured. Beware of data loss due to lack of redundant storage directories!
2018-10-21 23:42:35,668 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Edit logging is async:true
2018-10-21 23:42:35,683 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: KeyProvider: null
2018-10-21 23:42:35,685 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsLock is fair: true
2018-10-21 23:42:35,687 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Detailed lock hold time metrics enabled: false
2018-10-21 23:42:35,692 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner             = root (auth:SIMPLE)
2018-10-21 23:42:35,692 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup          = supergroup
2018-10-21 23:42:35,692 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isPermissionEnabled = true
2018-10-21 23:42:35,692 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Determined nameservice ID: dfsNameServices
2018-10-21 23:42:35,692 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: HA Enabled: true
2018-10-21 23:42:35,907 INFO org.apache.hadoop.hdfs.server.common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling
2018-10-21 23:42:35,938 INFO org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: dfs.block.invalidate.limit: configured=1000, counted=60, effected=1000
2018-10-21 23:42:35,938 INFO org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true
2018-10-21 23:42:35,941 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
2018-10-21 23:42:35,942 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: The block deletion will start around 2018 Oct 21 23:42:35
2018-10-21 23:42:35,947 INFO org.apache.hadoop.util.GSet: Computing capacity for map BlocksMap
2018-10-21 23:42:35,947 INFO org.apache.hadoop.util.GSet: VM type       = 64-bit
2018-10-21 23:42:35,981 INFO org.apache.hadoop.util.GSet: 2.0% max memory 966.7 MB = 19.3 MB
2018-10-21 23:42:35,981 INFO org.apache.hadoop.util.GSet: capacity      = 2^21 = 2097152 entries
2018-10-21 23:42:36,127 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: dfs.block.access.token.enable=false
2018-10-21 23:42:36,132 WARN org.apache.hadoop.conf.Configuration: No unit for dfs.namenode.safemode.extension(30000) assuming MILLISECONDS
2018-10-21 23:42:36,132 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
2018-10-21 23:42:36,132 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.min.datanodes = 0
2018-10-21 23:42:36,132 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.extension = 30000
2018-10-21 23:42:36,132 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: defaultReplication         = 3
2018-10-21 23:42:36,132 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxReplication             = 512
2018-10-21 23:42:36,132 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: minReplication             = 1
2018-10-21 23:42:36,132 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxReplicationStreams      = 2
2018-10-21 23:42:36,132 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: replicationRecheckInterval = 3000
2018-10-21 23:42:36,132 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: encryptDataTransfer        = false
2018-10-21 23:42:36,132 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxNumBlocksToLog          = 1000
2018-10-21 23:42:36,135 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Append Enabled: true
2018-10-21 23:42:37,355 INFO org.apache.hadoop.util.GSet: Computing capacity for map INodeMap
2018-10-21 23:42:37,355 INFO org.apache.hadoop.util.GSet: VM type       = 64-bit
2018-10-21 23:42:37,355 INFO org.apache.hadoop.util.GSet: 1.0% max memory 966.7 MB = 9.7 MB
2018-10-21 23:42:37,355 INFO org.apache.hadoop.util.GSet: capacity      = 2^20 = 1048576 entries
2018-10-21 23:42:37,370 INFO org.apache.hadoop.hdfs.server.namenode.FSDirectory: ACLs enabled? false
2018-10-21 23:42:37,371 INFO org.apache.hadoop.hdfs.server.namenode.FSDirectory: XAttrs enabled? true
2018-10-21 23:42:37,371 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Caching file names occurring more than 10 times
2018-10-21 23:42:37,375 INFO org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager: Loaded config captureOpenFiles: falseskipCaptureAccessTimeOnlyChange: false
2018-10-21 23:42:37,443 INFO org.apache.hadoop.util.GSet: Computing capacity for map cachedBlocks
2018-10-21 23:42:37,444 INFO org.apache.hadoop.util.GSet: VM type       = 64-bit
2018-10-21 23:42:37,444 INFO org.apache.hadoop.util.GSet: 0.25% max memory 966.7 MB = 2.4 MB
2018-10-21 23:42:37,444 INFO org.apache.hadoop.util.GSet: capacity      = 2^18 = 262144 entries
2018-10-21 23:42:37,447 INFO org.apache.hadoop.hdfs.server.namenode.top.metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10
2018-10-21 23:42:37,447 INFO org.apache.hadoop.hdfs.server.namenode.top.metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10
2018-10-21 23:42:37,447 INFO org.apache.hadoop.hdfs.server.namenode.top.metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25
2018-10-21 23:42:37,464 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Retry cache on namenode is enabled
2018-10-21 23:42:37,464 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
2018-10-21 23:42:37,465 INFO org.apache.hadoop.util.GSet: Computing capacity for map NameNodeRetryCache
2018-10-21 23:42:37,466 INFO org.apache.hadoop.util.GSet: VM type       = 64-bit
2018-10-21 23:42:37,466 INFO org.apache.hadoop.util.GSet: 0.029999999329447746% max memory 966.7 MB = 297.0 KB
2018-10-21 23:42:37,466 INFO org.apache.hadoop.util.GSet: capacity      = 2^15 = 32768 entries
2018-10-21 23:42:37,511 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /usr/local/hadoop_dir/dfs/name/in_use.lock acquired by nodename 2524@bigdata01
2018-10-21 23:42:39,412 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:39,413 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:39,436 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:40,423 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:40,436 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:40,439 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:41,434 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:41,446 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:41,446 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:42,436 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:42,454 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:42,454 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:43,437 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:43,455 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:43,456 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:44,107 INFO org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager: Waited 6001 ms (timeout=20000 ms) for a response for selectInputStreams. No responses yet.
2018-10-21 23:42:44,442 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:44,457 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:44,457 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:45,108 INFO org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager: Waited 7002 ms (timeout=20000 ms) for a response for selectInputStreams. No responses yet.
2018-10-21 23:42:45,444 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:45,458 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:45,459 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:46,109 INFO org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager: Waited 8003 ms (timeout=20000 ms) for a response for selectInputStreams. No responses yet.
2018-10-21 23:42:46,445 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:46,460 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:46,461 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:47,110 INFO org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager: Waited 9004 ms (timeout=20000 ms) for a response for selectInputStreams. No responses yet.
2018-10-21 23:42:47,454 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:47,466 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:47,466 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:48,115 INFO org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager: Waited 10009 ms (timeout=20000 ms) for a response for selectInputStreams. No responses yet.
2018-10-21 23:42:48,478 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:48,479 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:48,479 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:48,487 WARN org.apache.hadoop.hdfs.server.namenode.FSEditLog: Unable to determine input streams from QJM to [192.168.137.102:8485, 192.168.137.103:8485, 192.168.137.104:8485]. Skipping.
org.apache.hadoop.hdfs.qjournal.client.QuorumException: Got too many exceptions to achieve quorum size 2/3. 3 exceptions thrown:
192.168.137.103:8485: Call From bigdata01/192.168.137.101 to bigdata03:8485 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
192.168.137.104:8485: Call From bigdata01/192.168.137.101 to bigdata04:8485 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
192.168.137.102:8485: Call From bigdata01/192.168.137.101 to bigdata02:8485 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
	at org.apache.hadoop.hdfs.qjournal.client.QuorumException.create(QuorumException.java:81)
	at org.apache.hadoop.hdfs.qjournal.client.QuorumCall.rethrowException(QuorumCall.java:286)
	at org.apache.hadoop.hdfs.qjournal.client.AsyncLoggerSet.waitForWriteQuorum(AsyncLoggerSet.java:142)
	at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.selectInputStreams(QuorumJournalManager.java:473)
	at org.apache.hadoop.hdfs.server.namenode.JournalSet.selectInputStreams(JournalSet.java:278)
	at org.apache.hadoop.hdfs.server.namenode.FSEditLog.selectInputStreams(FSEditLog.java:1590)
	at org.apache.hadoop.hdfs.server.namenode.FSEditLog.selectInputStreams(FSEditLog.java:1614)
	at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:694)
	at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:316)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1048)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:681)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:666)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:728)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:953)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:932)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1673)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1741)
2018-10-21 23:42:48,490 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: No edit log streams selected.
2018-10-21 23:42:48,490 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Planning to load image: FSImageFile(file=/usr/local/hadoop_dir/dfs/name/current/fsimage_0000000000000028227, cpktTxId=0000000000000028227)
2018-10-21 23:42:48,664 INFO org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode: Loading 322 INodes.
2018-10-21 23:42:48,996 INFO org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf: Loaded FSImage in 0 seconds.
2018-10-21 23:42:48,996 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Loaded image for txid 28227 from /usr/local/hadoop_dir/dfs/name/current/fsimage_0000000000000028227
2018-10-21 23:42:49,042 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Need to save fs image? false (staleImage=true, haEnabled=true, isRollingUpgrade=false)
2018-10-21 23:42:49,043 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem write lock held for 11574 ms via
java.lang.Thread.getStackTrace(Thread.java:1559)
org.apache.hadoop.util.StringUtils.getStackTrace(StringUtils.java:1021)
org.apache.hadoop.hdfs.server.namenode.FSNamesystemLock.writeUnlock(FSNamesystemLock.java:252)
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.writeUnlock(FSNamesystem.java:1561)
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1077)
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:681)
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:666)
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:728)
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:953)
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:932)
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1673)
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1741)
	Number of suppressed write-lock reports: 0
	Longest write-lock held interval: 11574
2018-10-21 23:42:49,043 INFO org.apache.hadoop.hdfs.server.namenode.NameCache: initialized with 1 entries 11 lookups
2018-10-21 23:42:49,043 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Finished loading FSImage in 11574 msecs
2018-10-21 23:42:50,325 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: RPC server is binding to bigdata01:8020
2018-10-21 23:42:50,330 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue queueCapacity: 1000 scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler
2018-10-21 23:42:50,363 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 8020
2018-10-21 23:42:50,586 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered FSNamesystemState MBean
2018-10-21 23:42:50,660 INFO org.apache.hadoop.hdfs.server.namenode.LeaseManager: Number of blocks under construction: 0
2018-10-21 23:42:50,736 INFO org.apache.hadoop.hdfs.StateChange: STATE* Safe mode ON. 
The reported blocks 0 needs additional 182 blocks to reach the threshold 0.9990 of total blocks 183.
The number of live datanodes 0 has reached the minimum number 0. Safe mode will be turned off automatically once the thresholds have been reached.
2018-10-21 23:42:51,190 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: NameNode RPC up at: bigdata01/192.168.137.101:8020
2018-10-21 23:42:51,246 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Starting services required for standby state
2018-10-21 23:42:51,249 INFO org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer: Will roll logs on active node at bigdata02/192.168.137.102:8020 every 120 seconds.
2018-10-21 23:42:51,288 INFO org.apache.hadoop.hdfs.server.namenode.ha.StandbyCheckpointer: Starting standby checkpoint thread...
Checkpointing active NN at http://bigdata02:50070
Serving checkpoints at http://bigdata01:50070
2018-10-21 23:42:50,942 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2018-10-21 23:42:50,970 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8020: starting
2018-10-21 23:42:52,365 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:52,366 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:52,366 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:53,369 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:53,370 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:53,370 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:54,371 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:54,372 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:54,373 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:55,379 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:55,380 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:55,380 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:56,383 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:56,384 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:56,384 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:57,388 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:57,389 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:57,389 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:57,377 INFO org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager: Waited 6018 ms (timeout=20000 ms) for a response for selectInputStreams. No responses yet.
2018-10-21 23:42:58,389 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:58,390 INFO org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager: Waited 7031 ms (timeout=20000 ms) for a response for selectInputStreams. No responses yet.
2018-10-21 23:42:58,390 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:58,395 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:59,391 INFO org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager: Waited 8032 ms (timeout=20000 ms) for a response for selectInputStreams. No responses yet.
2018-10-21 23:42:59,394 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:59,394 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:42:59,397 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:00,398 INFO org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager: Waited 9039 ms (timeout=20000 ms) for a response for selectInputStreams. No responses yet.
2018-10-21 23:43:00,398 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:00,399 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:00,399 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:01,411 INFO org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager: Waited 10052 ms (timeout=20000 ms) for a response for selectInputStreams. No responses yet.
2018-10-21 23:43:01,411 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:01,411 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:01,412 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:01,416 WARN org.apache.hadoop.hdfs.server.namenode.FSEditLog: Unable to determine input streams from QJM to [192.168.137.102:8485, 192.168.137.103:8485, 192.168.137.104:8485]. Skipping.
org.apache.hadoop.hdfs.qjournal.client.QuorumException: Got too many exceptions to achieve quorum size 2/3. 3 exceptions thrown:
192.168.137.103:8485: Call From bigdata01/192.168.137.101 to bigdata03:8485 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
192.168.137.104:8485: Call From bigdata01/192.168.137.101 to bigdata04:8485 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
192.168.137.102:8485: Call From bigdata01/192.168.137.101 to bigdata02:8485 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
	at org.apache.hadoop.hdfs.qjournal.client.QuorumException.create(QuorumException.java:81)
	at org.apache.hadoop.hdfs.qjournal.client.QuorumCall.rethrowException(QuorumCall.java:286)
	at org.apache.hadoop.hdfs.qjournal.client.AsyncLoggerSet.waitForWriteQuorum(AsyncLoggerSet.java:142)
	at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.selectInputStreams(QuorumJournalManager.java:473)
	at org.apache.hadoop.hdfs.server.namenode.JournalSet.selectInputStreams(JournalSet.java:278)
	at org.apache.hadoop.hdfs.server.namenode.FSEditLog.selectInputStreams(FSEditLog.java:1590)
	at org.apache.hadoop.hdfs.server.namenode.FSEditLog.selectInputStreams(FSEditLog.java:1614)
	at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:244)
	at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:395)
	at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$300(EditLogTailer.java:348)
	at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:365)
	at org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:481)
	at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:361)
2018-10-21 23:43:01,417 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem write lock held for 10058 ms via
java.lang.Thread.getStackTrace(Thread.java:1559)
org.apache.hadoop.util.StringUtils.getStackTrace(StringUtils.java:1021)
org.apache.hadoop.hdfs.server.namenode.FSNamesystemLock.writeUnlock(FSNamesystemLock.java:252)
org.apache.hadoop.hdfs.server.namenode.FSNamesystemLock.writeUnlock(FSNamesystemLock.java:211)
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.writeUnlock(FSNamesystem.java:1558)
org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:278)
org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:395)
org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$300(EditLogTailer.java:348)
org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:365)
org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:481)
org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:361)
	Number of suppressed write-lock reports: 0
	Longest write-lock held interval: 10058
2018-10-21 23:43:01,425 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Stopping services started for standby state
2018-10-21 23:43:01,552 WARN org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer: Edit log tailer interrupted
java.lang.InterruptedException: sleep interrupted
	at java.lang.Thread.sleep(Native Method)
	at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:413)
	at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$300(EditLogTailer.java:348)
	at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:365)
	at org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:481)
	at org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:361)
2018-10-21 23:43:01,589 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Starting services required for active state
2018-10-21 23:43:01,633 INFO org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager: Starting recovery process for unclosed journal segments...
2018-10-21 23:43:02,716 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:02,730 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:02,730 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:03,737 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:03,738 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:03,738 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:04,741 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:04,741 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:04,741 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:05,753 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:05,767 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:05,767 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:06,754 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:06,768 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:06,769 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:07,762 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:07,777 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:07,777 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:08,771 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:08,783 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:08,783 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:09,792 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:09,792 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:09,792 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:10,795 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:10,795 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:10,795 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:11,797 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata03/192.168.137.103:8485. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:11,797 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata02/192.168.137.102:8485. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:11,797 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: bigdata04/192.168.137.104:8485. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2018-10-21 23:43:11,802 FATAL org.apache.hadoop.hdfs.server.namenode.FSEditLog: Error: recoverUnfinalizedSegments failed for required journal (JournalAndStream(mgr=QJM to [192.168.137.102:8485, 192.168.137.103:8485, 192.168.137.104:8485], stream=null))
org.apache.hadoop.hdfs.qjournal.client.QuorumException: Got too many exceptions to achieve quorum size 2/3. 3 exceptions thrown:
192.168.137.102:8485: Call From bigdata01/192.168.137.101 to bigdata02:8485 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
192.168.137.103:8485: Call From bigdata01/192.168.137.101 to bigdata03:8485 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
192.168.137.104:8485: Call From bigdata01/192.168.137.101 to bigdata04:8485 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
	at org.apache.hadoop.hdfs.qjournal.client.QuorumException.create(QuorumException.java:81)
	at org.apache.hadoop.hdfs.qjournal.client.QuorumCall.rethrowException(QuorumCall.java:286)
	at org.apache.hadoop.hdfs.qjournal.client.AsyncLoggerSet.waitForWriteQuorum(AsyncLoggerSet.java:142)
	at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.createNewUniqueEpoch(QuorumJournalManager.java:180)
	at org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager.recoverUnfinalizedSegments(QuorumJournalManager.java:438)
	at org.apache.hadoop.hdfs.server.namenode.JournalSet$8.apply(JournalSet.java:624)
	at org.apache.hadoop.hdfs.server.namenode.JournalSet.mapJournalsAndReportErrors(JournalSet.java:393)
	at org.apache.hadoop.hdfs.server.namenode.JournalSet.recoverUnfinalizedSegments(JournalSet.java:621)
	at org.apache.hadoop.hdfs.server.namenode.FSEditLog.recoverUnclosedStreams(FSEditLog.java:1521)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startActiveServices(FSNamesystem.java:1176)
	at org.apache.hadoop.hdfs.server.namenode.NameNode$NameNodeHAContext.startActiveServices(NameNode.java:1919)
	at org.apache.hadoop.hdfs.server.namenode.ha.ActiveState.enterState(ActiveState.java:61)
	at org.apache.hadoop.hdfs.server.namenode.ha.HAState.setStateInternal(HAState.java:64)
	at org.apache.hadoop.hdfs.server.namenode.ha.StandbyState.setState(StandbyState.java:49)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.transitionToActive(NameNode.java:1777)
	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.transitionToActive(NameNodeRpcServer.java:1649)
	at org.apache.hadoop.ha.protocolPB.HAServiceProtocolServerSideTranslatorPB.transitionToActive(HAServiceProtocolServerSideTranslatorPB.java:107)
	at org.apache.hadoop.ha.proto.HAServiceProtocolProtos$HAServiceProtocolService$2.callBlockingMethod(HAServiceProtocolProtos.java:4460)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:503)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:868)
	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:814)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1886)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2603)
2018-10-21 23:43:11,805 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1: Error: recoverUnfinalizedSegments failed for required journal (JournalAndStream(mgr=QJM to [192.168.137.102:8485, 192.168.137.103:8485, 192.168.137.104:8485], stream=null))
2018-10-21 23:43:11,809 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at bigdata01/192.168.137.101
************************************************************/

以上是NameNode启动日志，先记录下。

hadoop集群之前好好的，有一天启动，发现NameNode无法启动。在大神的一篇文章中顺利解决

https://www.linuxidc.com/Linux/2016-03/129437.htm

在此记录下自己踩过的坑，写个备忘录。

1. 在【core-site.xml】文件中，添加以下内容（core-default.xml）：

<property>
  <name>ipc.client.connect.max.retries</name>
  <value>100</value>
</property>
<property>
  <name>ipc.client.connect.retry.interval</name>
  <value>10000</value>
</property>

2. 分发此文件到集群其他节点。

转载于:https://my.oschina.net/zxdzyxzx/blog/2250765

chengzhui0990
关注
Hadoop Namenode无法启动-数据节点名称不一致问题
GnabVue的博客
10-06 335
但是，有时候你可能会遇到Namenode无法启动的问题，并且错误信息中提到"dfs name is in an inconsistent"（数据节点名称不一致）。当Namenode启动时，它会检查数据节点的注册信息，以确保每个数据节点的名称都是唯一且一致的。如果它发现有两个或多个数据节点具有相同的名称，或者数据节点的名称与Namenode的记录不匹配，就会出现数据节点名称不一致的错误。解决这个问题的一种常见方法是手动清除数据节点的注册信息，并重新启动Hadoop集群。步骤 2：删除数据节点注册信息。
Hadoop--万恶NameNode各种起不来！！！
樱桃气质丸的博客
03-20 2621
一下↓ 是对NameNode结点一些常见问题这里用到启动Hadoop集群的脚本：myhadoop.sh start 和查看进程脚本：jpsall (后面会写一篇关于脚本的文章)先看Hadoop配置文件/opt/module/hadoop-3.1.3/etc/hadoop（每个人可能路径不同），查看这个目录下的core-site.xml 、 yarn-site.xml 、 hdfs-site.xml 、mapred-site.xml 、 workers仔细阅读配置文件，字母、括号都不要错过！！！！
Hadoop HA HDFS启动错误之org.apache.hadoop.ipc.Client: Retrying connect to server问题解决
weixin_33949359的博客
03-19 1912
近日，在搭建Hadoop HA QJM集群的时候，出现一个问题，如本文标题。网上有很多HA的博文，其实比较好的博文就是官方文档，讲的已经非常详细。所以，HA的搭建这里不再赘述。本文就想给出一篇org.apache.hadoop.ipc.Client: Retrying connect to server错误的解决的方法。因为在搜索引擎中输入了错误问题，没有找到一篇解决问题的。...
hadoop启动没有namenode节点的错误分析
风雨诗轩的博客
07-11 5281
我用的是hadoop-1.0.4，伪分布式。没有为hadoop新建用户，我直接使用root启动，当输入#hadoop namenode -format后，也没什么异常，继续往下执行， 输入#start-all.sh后，启动完后输入#jps发现没有namenode，立马去看日志，显示如下： 2017-07-11 14:18:47,407 INFO org.apache.hadoop.hdfs
hadoop 的HDFS 的 standby namenode无法启动事故处理
weixin_30435261的博客
01-22 877
standby namenode无法启动 现象：线上使用的2.5.0-cdh5.3.2版本Hadoop，开启了了NameNode HA，HA采用QJM方式。hadoop的集群的namenode的standby节点宕掉，重启无法启动成功。 1. standby namenode出现异常时的现象 异常现象1: hadoop的集群的namenode的standby节点宕掉，重启无法启动成功。查看ha...
WARN org.apache.hadoop.hdfs.servåer.namenode.FSNamesystem: Get corrupt file b
wulantian的专栏
06-23 7021
Hadoop运维笔记 之 Namenode异常停止后无法正常启动： 存储，学习，共享。。。。 公司在线上使用了CDH5 HA模式，有两个Namenode节点，结果其中的Standby节点因为一些关于edits文件的报错异常停止了，并且在启动的过程中一直报告找不到各种文件。 刚开始怀疑问题可能只发生在Standby本身，因此尝试了bootstrapStandby
hadoop错误一，namenode启动问题
weixin_34218890的博客
03-17 181
hadoop错误汇总问题：环境中的hadoop因为一些原因需要重启，在使用stop-all.sh和start-all.sh命令重启后，出现了，hadoop集群无法访问，hadoop的web页面也无法打开。查看日志还有namenode的端口后，发现hadoop的namenode节点无法启动成功。出现一下错误。2013-03-16 10:59:30,857 INFO org.ap...
Hadoop Namenode恢复
最新发布
09-02
三、重启 Hadoop 并检查 namenode 的启动情况 在修改完配置文件后，需要重启 Hadoop，检查 namenode 是否启动成功。可以使用 jps 命令查看 namenode 的进程。 四、恢复 namenode 在 namenode 崩溃的情况下，需要...
教你一步一步分析Hadoop NameNode启动流程
shalistone的博客
05-06 651
NameNode服务端的启动过程 不知道大家是否自己搭建过Hadoop集群，如果搭建过是否还记得Hadoop NameNode的启动命令呢？下面是Linux服务器上启动NameNode的命令。 [root@hadoop01 sbin]# hadoop-daemon.sh start namenode 既然是通过hadoop-daemon.s进行启动，那么我们可以看看这个shell脚本是怎么启...
Hadoop Namenode启动报错GC overhead limit exceeded
nickyu888的博客
02-15 600
参考： https://blog.csdn.net/rongyongfeikai2/article/details/82878578 文中所述，由于fsimage的inodes过多，造成启动时内存不够用所致。 需要用以下命令查看fsimage文件占用的内存大小： ./hdfs oiv -p XML -printToScreen -i ${fsimage文件路径} -o /tmp/a cat /...
hadoop集群的namenode无法启动
热门推荐
weixin_41596463的博客
06-05 1万+
1、首先namenode无法启动，第一时间查看日志，看看具体是什么原因。2、namenode无法启动，主要还是配置文件的错误，其次就是用户权限的问题。3、日志错误如下：2018-06-02 23:57:20,316 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG: /************************...
HA模式下，扩展HDFS集群遇到的问题，namenode 启动不起来！emmmmmmmm~
qq_31806205的博客
09-06 2374
HA模式下，扩展HDFS集群遇到的问题，namenode 启动不起来！ Journal Storage Directory /hadoop/journaldata/cluster not formatted Operation category JOURNAL is not supported in state standby
Hadoop学习-错误记录：namenode、datanode、secondarynamenode未启动
摘果子的人的博客
09-02 1486
1、在ubuntu 16.04系统中，采用伪分布配置，在账户A上启动hadoop 2.7.3失败，namenode、datanode、secondarynamenode未启动，查看hadoop-hadoop-namenode-ubuntu.log，错误信息如下： 2017-09-02 02:33:30,597 INFO org.apache.hadoop.http.HttpServer2: Htt
hadoop namenode启动异常，死活失败
weixin_34126557的博客
05-12 168
2014-05-12注定是春光灿烂猪八戒的一天，历史595无故障的hadoop服务器，终于还是出了问题，事前无人登陆操作服务器，此故障属于自发行为，目前未知发生原因。细节描述： namenode无法启动. 先贴出错误信息2014-05-12 07:17:39,447 INFO org.apache.hadoop.hdfs.server.namenode.NameNod...
hadoop2.6.0 namenode格式化报错
shouwangV6的专栏
05-13 1995
今天在安装hadoop2.6.0时格式化namenode时报错 15/05/12 18:28:19 INFO ipc.Client: Retrying connect to server: hadoop3/192.168.110.139:8485. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSle
hadoop格式化失败提示lostconnection
qq_43662627的博客
08-13 402
可能是hadoop的日志文件没有清空, 在自己的Hadoop目录下的core-site.xml中配置的路径为 /export/servers/hadoop/tmp就应该在格式化前将此路径下的所有文件删除
cdh namenode异常停止，报错：Error: flush failed for required journal (JournalAndStream(mgr=QJM to
h952520296的博客
06-18 1400
hadoop集群主Namenode突然停止，报错如下： 2016-03-23 17:12:25,877 INFOnamenode.FSEditLog (FSEditLog.java:endCurrentLogSegment(1153)) - Ending log segment 574144342 2016-03-23 17:12:26,350 WARNclient.QuorumJournalManager (QuorumCall.java:waitFor(134)) - Waited 19047 ...
Hadoop集群namenode(standby)，异常挂掉问题
mnasd的博客
12-11 4538
2月22号，发现namenode02服务器的namenode(standby)节点挂掉，查看hadoop日志/app/hadoop/logs/hadoop-appadm-namenode-prd-bldb-hdp-name02.log 发现2018-02-17 03:29:34，首次报出java.lang.OutOfMemoryError的ERROR，具体报错信息如下   2018-02...
【Hadoop】运维记录
u013202518的博客
04-04 1470
1. namenode 被重新格式化，datanode数据版本与namenode不一致 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Incompatible namespaceIDs in /home/hadoop/tmp/dfs/data: namenode namespaceID = 39895076; datanode namespaceID = 1030326122 删除datanode
hadoop namenode无法启动
03-17
Hadoop Namenode 无法启动可能是由于多种原因造成的。首先，您可以检查日志文件以了解具体的错误信息。其次，您可以检查配置文件是否正确。如果文件系统已损坏，您可能需要进行文件系统修复。如果是硬件问题，你可能...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值