Hadoop 2.4.1 搭建Ha遇到问题记录

原创 2015年07月08日 22:35:00
15/06/19 17:12:44 ERROR namenode.FSNamesystem: FSNamesystem initialization failed.
java.io.IOException: Invalid configuration: <span style="background-color: rgb(255, 102, 102);">a shared edits dir must not be specified if HA is not enabled</span>. // 明明我配置了HA,却说没有,下面的配置写错了
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:710)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:654)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:892)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1310)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1424)
15/06/19 17:12:44 INFO namenode.FSNamesystem: Stopping services started for active state
15/06/19 17:12:44 FATAL namenode.NameNode: Exception in namenode join
java.lang.NullPointerException
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.stopActiveServices(FSNamesystem.java:1118)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.close(FSNamesystem.java:1446)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:781)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:654)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:892)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1310)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1424)
15/06/19 17:12:44 INFO util.ExitUtil: Exiting with status 1
15/06/19 17:12:44 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at v1/192.168.32.91
************************************************************/


<strong>.写成了-</strong>
<property>
                <name>dfs.ha.automatic-failover-enabled</name>
                <value>true</value>
        </property>
正确如下:
<property>
                <name>dfs.ha.automatic-failover.enabled</name>
                <value>true</value>
        </property>


2.journalnode的端口配置错误,导致不能格式化namenode,journalnode监听的端口是:8485,而不是8445.
3.格式化standby namnenode时,有两种方式
    1.将active namenode格式化完成后,将整个hadoop.tmp.dir路径全部拷贝到standby机器上。
    2.在active namenode格式化完成后,并且格式化zkfc(只需要在active上格式化一次,standby上的zkfc不需要再次格式化)启动好了namenode之后。 在standby namenode上执行:hdfs namenode -bootstrapStandby 也可以.如果active namenode没有启动的话,这里会报错。因为standby namenode是通过active namenode的9000端口通讯的。若active namenode没有启动,则9000没有程序监听提供服务,standby尝试连接9次后会爆异常并且退出。

没有启动active namenode时,报错如下:
[root@v2 hadoop]# hdfs namenode -bootstrapStandby
15/06/19 19:28:45 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = v2/192.168.32.92
STARTUP_MSG:   args = [-bootstrapStandby]
STARTUP_MSG:   version = 2.4.1
STARTUP_MSG:   classpath = /root/hadoop/etc/hadoop:/root/hadoop/share/hadoop/common/lib/asm-3.2.jar:/root/hadoop/share/hadoop/common/lib/xz-1.0.jar:/root/hadoop/share/hadoop/common/lib/junit-4.8.2.jar:/root/hadoop/share/hadoop/common/lib/httpcore-4.2.5.jar:/root/hadoop/share/hadoop/common/lib/jetty-6.1.26.jar:/root/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/root/hadoop/share/hadoop/common/lib/jetty-util-6.1.26.jar:/root/hadoop/share/hadoop/common/lib/commons-compress-1.4.1.jar:/root/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/common/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/common/lib/hadoop-auth-2.4.1.jar:/root/hadoop/share/hadoop/common/lib/stax-api-1.0-2.jar:/root/hadoop/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/root/hadoop/share/hadoop/common/lib/netty-3.6.2.Final.jar:/root/hadoop/share/hadoop/common/lib/httpclient-4.2.5.jar:/root/hadoop/share/hadoop/common/lib/jsr305-1.3.9.jar:/root/hadoop/share/hadoop/common/lib/avro-1.7.4.jar:/root/hadoop/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/root/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/common/lib/commons-codec-1.4.jar:/root/hadoop/share/hadoop/common/lib/commons-el-1.0.jar:/root/hadoop/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/root/hadoop/share/hadoop/common/lib/guava-11.0.2.jar:/root/hadoop/share/hadoop/common/lib/xmlenc-0.52.jar:/root/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/root/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/root/hadoop/share/hadoop/common/lib/mockito-all-1.8.5.jar:/root/hadoop/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/root/hadoop/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/root/hadoop/share/hadoop/common/lib/jsch-0.1.42.jar:/root/hadoop/share/hadoop/common/lib/hadoop-annotations-2.4.1.jar:/root/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/root/hadoop/share/hadoop/common/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/common/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar:/root/hadoop/share/hadoop/common/lib/jets3t-0.9.0.jar:/root/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/root/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/root/hadoop/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/root/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/root/hadoop/share/hadoop/common/lib/jersey-json-1.9.jar:/root/hadoop/share/hadoop/common/lib/commons-httpclient-3.1.jar:/root/hadoop/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/root/hadoop/share/hadoop/common/lib/commons-collections-3.2.1.jar:/root/hadoop/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/root/hadoop/share/hadoop/common/lib/activation-1.1.jar:/root/hadoop/share/hadoop/common/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/root/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar:/root/hadoop/share/hadoop/common/lib/commons-digester-1.8.jar:/root/hadoop/share/hadoop/common/lib/servlet-api-2.5.jar:/root/hadoop/share/hadoop/common/lib/commons-lang-2.6.jar:/root/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/root/hadoop/share/hadoop/common/lib/zookeeper-3.4.5.jar:/root/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/root/hadoop/share/hadoop/common/hadoop-nfs-2.4.1.jar:/root/hadoop/share/hadoop/common/hadoop-common-2.4.1-tests.jar:/root/hadoop/share/hadoop/common/hadoop-common-2.4.1.jar:/root/hadoop/share/hadoop/hdfs:/root/hadoop/share/hadoop/hdfs/lib/asm-3.2.jar:/root/hadoop/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/root/hadoop/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/root/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/root/hadoop/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/root/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-el-1.0.jar:/root/hadoop/share/hadoop/hdfs/lib/guava-11.0.2.jar:/root/hadoop/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/root/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/root/hadoop/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/root/hadoop/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/root/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/root/hadoop/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/root/hadoop/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/root/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-2.4.1.jar:/root/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.4.1.jar:/root/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.4.1-tests.jar:/root/hadoop/share/hadoop/yarn/lib/asm-3.2.jar:/root/hadoop/share/hadoop/yarn/lib/xz-1.0.jar:/root/hadoop/share/hadoop/yarn/lib/jetty-6.1.26.jar:/root/hadoop/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/root/hadoop/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/root/hadoop/share/hadoop/yarn/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/yarn/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-xc-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/jsr305-1.3.9.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/commons-codec-1.4.jar:/root/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar:/root/hadoop/share/hadoop/yarn/lib/jline-0.9.94.jar:/root/hadoop/share/hadoop/yarn/lib/guava-11.0.2.jar:/root/hadoop/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/root/hadoop/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/root/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/root/hadoop/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-json-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/root/hadoop/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/root/hadoop/share/hadoop/yarn/lib/activation-1.1.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/root/hadoop/share/hadoop/yarn/lib/commons-cli-1.2.jar:/root/hadoop/share/hadoop/yarn/lib/servlet-api-2.5.jar:/root/hadoop/share/hadoop/yarn/lib/commons-lang-2.6.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-client-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/jettison-1.1.jar:/root/hadoop/share/hadoop/yarn/lib/zookeeper-3.4.5.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/guice-3.0.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-api-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-client-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/asm-3.2.jar:/root/hadoop/share/hadoop/mapreduce/lib/xz-1.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/hamcrest-core-1.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/root/hadoop/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/root/hadoop/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/root/hadoop/share/hadoop/mapreduce/lib/javax.inject-1.jar:/root/hadoop/share/hadoop/mapreduce/lib/hadoop-annotations-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/junit-4.10.jar:/root/hadoop/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/root/hadoop/share/hadoop/mapreduce/lib/guice-3.0.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.4.1-tests.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.4.1.jar:/root/hadoop/contrib/capacity-scheduler/*.jar
STARTUP_MSG:   build = http://svn.apache.org/repos/asf/hadoop/common -r 1604318; compiled by 'jenkins' on 2014-06-21T05:43Z
STARTUP_MSG:   java = 1.7.0_15
************************************************************/
15/06/19 19:28:45 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
15/06/19 19:28:45 INFO namenode.NameNode: createNameNode [-bootstrapStandby]
15/06/19 19:28:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/06/19 19:28:49 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/06/19 19:28:50 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/06/19 19:28:51 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/06/19 19:28:52 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/06/19 19:28:53 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/06/19 19:28:54 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/06/19 19:28:55 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/06/19 19:28:56 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/06/19 19:28:57 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/06/19 19:28:58 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/06/19 19:28:58 FATAL ha.BootstrapStandby: Unable to fetch namespace information from active NN at v1/192.168.32.91:9000: Call From v2/192.168.32.92 to v1:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
15/06/19 19:28:58 INFO util.ExitUtil: Exiting with status 2
15/06/19 19:28:58 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at v2/192.168.32.92
************************************************************/

启动了active如下:

[root@v2 hadoop]# hdfs namenode -bootstrapStandby
15/06/19 19:31:39 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = v2/192.168.32.92
STARTUP_MSG:   args = [-bootstrapStandby]
STARTUP_MSG:   version = 2.4.1
STARTUP_MSG:   classpath = /root/hadoop/etc/hadoop:/root/hadoop/share/hadoop/common/lib/asm-3.2.jar:/root/hadoop/share/hadoop/common/lib/xz-1.0.jar:/root/hadoop/share/hadoop/common/lib/junit-4.8.2.jar:/root/hadoop/share/hadoop/common/lib/httpcore-4.2.5.jar:/root/hadoop/share/hadoop/common/lib/jetty-6.1.26.jar:/root/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/root/hadoop/share/hadoop/common/lib/jetty-util-6.1.26.jar:/root/hadoop/share/hadoop/common/lib/commons-compress-1.4.1.jar:/root/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/common/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/common/lib/hadoop-auth-2.4.1.jar:/root/hadoop/share/hadoop/common/lib/stax-api-1.0-2.jar:/root/hadoop/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/root/hadoop/share/hadoop/common/lib/netty-3.6.2.Final.jar:/root/hadoop/share/hadoop/common/lib/httpclient-4.2.5.jar:/root/hadoop/share/hadoop/common/lib/jsr305-1.3.9.jar:/root/hadoop/share/hadoop/common/lib/avro-1.7.4.jar:/root/hadoop/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/root/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/common/lib/commons-codec-1.4.jar:/root/hadoop/share/hadoop/common/lib/commons-el-1.0.jar:/root/hadoop/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/root/hadoop/share/hadoop/common/lib/guava-11.0.2.jar:/root/hadoop/share/hadoop/common/lib/xmlenc-0.52.jar:/root/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/root/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/root/hadoop/share/hadoop/common/lib/mockito-all-1.8.5.jar:/root/hadoop/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/root/hadoop/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/root/hadoop/share/hadoop/common/lib/jsch-0.1.42.jar:/root/hadoop/share/hadoop/common/lib/hadoop-annotations-2.4.1.jar:/root/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/root/hadoop/share/hadoop/common/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/common/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar:/root/hadoop/share/hadoop/common/lib/jets3t-0.9.0.jar:/root/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/root/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/root/hadoop/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/root/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/root/hadoop/share/hadoop/common/lib/jersey-json-1.9.jar:/root/hadoop/share/hadoop/common/lib/commons-httpclient-3.1.jar:/root/hadoop/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/root/hadoop/share/hadoop/common/lib/commons-collections-3.2.1.jar:/root/hadoop/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/root/hadoop/share/hadoop/common/lib/activation-1.1.jar:/root/hadoop/share/hadoop/common/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/root/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar:/root/hadoop/share/hadoop/common/lib/commons-digester-1.8.jar:/root/hadoop/share/hadoop/common/lib/servlet-api-2.5.jar:/root/hadoop/share/hadoop/common/lib/commons-lang-2.6.jar:/root/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/root/hadoop/share/hadoop/common/lib/zookeeper-3.4.5.jar:/root/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/root/hadoop/share/hadoop/common/hadoop-nfs-2.4.1.jar:/root/hadoop/share/hadoop/common/hadoop-common-2.4.1-tests.jar:/root/hadoop/share/hadoop/common/hadoop-common-2.4.1.jar:/root/hadoop/share/hadoop/hdfs:/root/hadoop/share/hadoop/hdfs/lib/asm-3.2.jar:/root/hadoop/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/root/hadoop/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/root/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/root/hadoop/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/root/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-el-1.0.jar:/root/hadoop/share/hadoop/hdfs/lib/guava-11.0.2.jar:/root/hadoop/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/root/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/root/hadoop/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/root/hadoop/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/root/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/root/hadoop/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/root/hadoop/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/root/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-2.4.1.jar:/root/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.4.1.jar:/root/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.4.1-tests.jar:/root/hadoop/share/hadoop/yarn/lib/asm-3.2.jar:/root/hadoop/share/hadoop/yarn/lib/xz-1.0.jar:/root/hadoop/share/hadoop/yarn/lib/jetty-6.1.26.jar:/root/hadoop/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/root/hadoop/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/root/hadoop/share/hadoop/yarn/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/yarn/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-xc-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/jsr305-1.3.9.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/commons-codec-1.4.jar:/root/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar:/root/hadoop/share/hadoop/yarn/lib/jline-0.9.94.jar:/root/hadoop/share/hadoop/yarn/lib/guava-11.0.2.jar:/root/hadoop/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/root/hadoop/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/root/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/root/hadoop/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-json-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/root/hadoop/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/root/hadoop/share/hadoop/yarn/lib/activation-1.1.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/root/hadoop/share/hadoop/yarn/lib/commons-cli-1.2.jar:/root/hadoop/share/hadoop/yarn/lib/servlet-api-2.5.jar:/root/hadoop/share/hadoop/yarn/lib/commons-lang-2.6.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-client-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/jettison-1.1.jar:/root/hadoop/share/hadoop/yarn/lib/zookeeper-3.4.5.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/guice-3.0.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-api-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-client-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/asm-3.2.jar:/root/hadoop/share/hadoop/mapreduce/lib/xz-1.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/hamcrest-core-1.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/root/hadoop/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/root/hadoop/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/root/hadoop/share/hadoop/mapreduce/lib/javax.inject-1.jar:/root/hadoop/share/hadoop/mapreduce/lib/hadoop-annotations-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/junit-4.10.jar:/root/hadoop/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/root/hadoop/share/hadoop/mapreduce/lib/guice-3.0.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.4.1-tests.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.4.1.jar:/root/hadoop/contrib/capacity-scheduler/*.jar
STARTUP_MSG:   build = http://svn.apache.org/repos/asf/hadoop/common -r 1604318; compiled by 'jenkins' on 2014-06-21T05:43Z
STARTUP_MSG:   java = 1.7.0_15
************************************************************/
15/06/19 19:31:39 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
15/06/19 19:31:39 INFO namenode.NameNode: createNameNode [-bootstrapStandby]
15/06/19 19:31:41 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
=====================================================
About to bootstrap Standby ID bnn2 from:
           Nameservice ID: bns1
        Other Namenode ID: bnn1
  Other NN's HTTP address: http://v1:50070
  Other NN's IPC  address: v1/192.168.32.91:9000
             Namespace ID: 32897138
            Block pool ID: BP-134496651-192.168.32.91-1434751114027
               Cluster ID: CID-c5890502-45a9-425e-9ae3-aa70c3eff8a6
           Layout version: -56
=====================================================
15/06/19 19:31:44 INFO common.Storage: Storage directory /root/hadoop/tmp/dfs/name has been successfully formatted.
15/06/19 19:31:45 INFO namenode.TransferFsImage: Opening connection to http://v1:50070/imagetransfer?getimage=1&txid=0&storageInfo=-56:32897138:0:CID-c5890502-45a9-425e-9ae3-aa70c3eff8a6
15/06/19 19:31:46 INFO namenode.TransferFsImage: Image Transfer timeout configured to 60000 milliseconds
15/06/19 19:31:46 INFO namenode.TransferFsImage: Transfer took 0.01s at 0.00 KB/s
15/06/19 19:31:46 INFO namenode.TransferFsImage: Downloaded file fsimage.ckpt_0000000000000000000 size 350 bytes.
15/06/19 19:31:46 INFO util.ExitUtil: Exiting with status 0
15/06/19 19:31:46 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at v2/192.168.32.92
************************************************************/


4.hbase启动时,如果不同节点时间不同的话,启动只有HMaster会起来,别的HRegionServer都会挂掉。同步时间即可,HBase需要用到hadoop的{core-site.xml/hdfs-site.xml},否则会将配置好的ha hdfsname(hdfs://bns1/hbase)当成hostname去解析,把hdfs相关文件拷贝到hbase/conf下就能正常运行了。
5.hive配置元数据快记法:
    4个配置项前缀:javax.jdo.option.Connection[URL/UserName/Password/DriverName]
    createDatabaseIfNotExist
    配置mysql允许远程连接:
        #(执行下面的语句  *.*:所有库下的所有表   %:任何IP地址或主机都可以连接)
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123' WITH GRANT OPTION;
FLUSH PRIVILEGES;
        如果这么配置完了之后,可以远程登录但是本机登录不了的时候。可以用mysqladmin给本地重新修改下密码
        /usr/bin/mysqladmin -uroot -h currentHostName password 'new-password'


HA 模式下的 Hadoop+ZooKeeper+HBase 启动顺序

2 主机规划 一 首次启动集群 网络上的大部分教程的顺序 启动顺序 停止顺序 1.2 主机规划 IP 主机名 用户名 部署模块 进程 1...
  • u011414200
  • u011414200
  • 2016年01月01日 11:31
  • 9230

搭建hdfsHA时遇到的问题及其解决方案

问题:1、        ERROR namenode.FSNamesystem: FSNamesystem initialization failed. java.io.IOExceptio...
  • u013226462
  • u013226462
  • 2015年09月30日 15:31
  • 2620

hadoop执行hdfs namenode -bootstrapStandby报错FATAL ha.BootstrapStandby的解决方法

在执行hdfs namenode -bootstrapStandby的时候,一直报错,  FATAL ha.BootstrapStandby: Unable to fetch namespace in...
  • qq236237606
  • qq236237606
  • 2017年04月13日 14:18
  • 408

Zookeeper

====zookeeper============================================== ZooKeeper  -- 动物园管理员     hadoop -- 大...
  • Helpless_pain
  • Helpless_pain
  • 2017年04月13日 09:19
  • 351

hadoop环境搭建

前记 最近开始有时间折腾hadoop了,所以打算系统的学习一下,并做一些纪录。首先从环境搭建开始。本人使用hadoop 2.4.0版本。 虚拟机安装 本人只有两台电脑,貌似不够用,只能靠...
  • DLevin
  • DLevin
  • 2014年06月10日 21:29
  • 4341

hadoop架构部署详细方案

  • 2014年03月12日 15:04
  • 7KB
  • 下载

集群搭建小结

主备节点启动小结在这次ha集群的搭建过程中细节上出现了较多卡顿。 针对搭建好后的主备节点的启用也出现了一些问题。 在启动时单在一个节点上格式化后,另一个namenode节点并未同步格式化内容 可...
  • wqkenqing
  • wqkenqing
  • 2017年02月27日 12:26
  • 82

Hadoop必备

论坛: zookeeper  :  http://www.aboutyun.com/forum-149-1.html hadoop  :  http://www.aboutyun.com/foru...
  • qq_36864672
  • qq_36864672
  • 2017年10月27日 21:27
  • 142

Hadoop安装遇到的各种异常及解决办法(1)

异常一: 2014-03-13 11:10:23,665 INFO org.apache.Hadoop.ipc.Client: Retrying connect to server: Linux-h...
  • lifuxiangcaohui
  • lifuxiangcaohui
  • 2014年09月14日 14:02
  • 23220

Hadoop 2.6.0(HA)部署启动

如果是一个新的HDFS集群,还要首先执行格式化命令“hdfs namenode -format”,紧接着启动本NameNode进程。 如果存在一个已经格式化过的NameNode,并且已经启动了。那么应...
  • jsky_studio
  • jsky_studio
  • 2015年08月16日 10:16
  • 571
内容举报
返回顶部
收藏助手
不良信息举报
您举报文章:Hadoop 2.4.1 搭建Ha遇到问题记录
举报原因:
原因补充:

(最多只允许输入30个字)