关闭

Hadoop 2.4.1 搭建Ha遇到问题记录

标签: hadoophbasehdfs
3564人阅读 评论(0) 收藏 举报
分类:
15/06/19 17:12:44 ERROR namenode.FSNamesystem: FSNamesystem initialization failed.
java.io.IOException: Invalid configuration: <span style="background-color: rgb(255, 102, 102);">a shared edits dir must not be specified if HA is not enabled</span>. // 明明我配置了HA,却说没有,下面的配置写错了
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:710)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:654)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:892)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1310)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1424)
15/06/19 17:12:44 INFO namenode.FSNamesystem: Stopping services started for active state
15/06/19 17:12:44 FATAL namenode.NameNode: Exception in namenode join
java.lang.NullPointerException
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.stopActiveServices(FSNamesystem.java:1118)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.close(FSNamesystem.java:1446)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:781)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:654)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:892)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1310)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1424)
15/06/19 17:12:44 INFO util.ExitUtil: Exiting with status 1
15/06/19 17:12:44 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at v1/192.168.32.91
************************************************************/


<strong>.写成了-</strong>
<property>
                <name>dfs.ha.automatic-failover-enabled</name>
                <value>true</value>
        </property>
正确如下:
<property>
                <name>dfs.ha.automatic-failover.enabled</name>
                <value>true</value>
        </property>


2.journalnode的端口配置错误,导致不能格式化namenode,journalnode监听的端口是:8485,而不是8445.
3.格式化standby namnenode时,有两种方式
    1.将active namenode格式化完成后,将整个hadoop.tmp.dir路径全部拷贝到standby机器上。
    2.在active namenode格式化完成后,并且格式化zkfc(只需要在active上格式化一次,standby上的zkfc不需要再次格式化)启动好了namenode之后。 在standby namenode上执行:hdfs namenode -bootstrapStandby 也可以.如果active namenode没有启动的话,这里会报错。因为standby namenode是通过active namenode的9000端口通讯的。若active namenode没有启动,则9000没有程序监听提供服务,standby尝试连接9次后会爆异常并且退出。

没有启动active namenode时,报错如下:
[root@v2 hadoop]# hdfs namenode -bootstrapStandby
15/06/19 19:28:45 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = v2/192.168.32.92
STARTUP_MSG:   args = [-bootstrapStandby]
STARTUP_MSG:   version = 2.4.1
STARTUP_MSG:   classpath = /root/hadoop/etc/hadoop:/root/hadoop/share/hadoop/common/lib/asm-3.2.jar:/root/hadoop/share/hadoop/common/lib/xz-1.0.jar:/root/hadoop/share/hadoop/common/lib/junit-4.8.2.jar:/root/hadoop/share/hadoop/common/lib/httpcore-4.2.5.jar:/root/hadoop/share/hadoop/common/lib/jetty-6.1.26.jar:/root/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/root/hadoop/share/hadoop/common/lib/jetty-util-6.1.26.jar:/root/hadoop/share/hadoop/common/lib/commons-compress-1.4.1.jar:/root/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/common/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/common/lib/hadoop-auth-2.4.1.jar:/root/hadoop/share/hadoop/common/lib/stax-api-1.0-2.jar:/root/hadoop/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/root/hadoop/share/hadoop/common/lib/netty-3.6.2.Final.jar:/root/hadoop/share/hadoop/common/lib/httpclient-4.2.5.jar:/root/hadoop/share/hadoop/common/lib/jsr305-1.3.9.jar:/root/hadoop/share/hadoop/common/lib/avro-1.7.4.jar:/root/hadoop/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/root/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/common/lib/commons-codec-1.4.jar:/root/hadoop/share/hadoop/common/lib/commons-el-1.0.jar:/root/hadoop/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/root/hadoop/share/hadoop/common/lib/guava-11.0.2.jar:/root/hadoop/share/hadoop/common/lib/xmlenc-0.52.jar:/root/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/root/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/root/hadoop/share/hadoop/common/lib/mockito-all-1.8.5.jar:/root/hadoop/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/root/hadoop/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/root/hadoop/share/hadoop/common/lib/jsch-0.1.42.jar:/root/hadoop/share/hadoop/common/lib/hadoop-annotations-2.4.1.jar:/root/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/root/hadoop/share/hadoop/common/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/common/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar:/root/hadoop/share/hadoop/common/lib/jets3t-0.9.0.jar:/root/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/root/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/root/hadoop/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/root/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/root/hadoop/share/hadoop/common/lib/jersey-json-1.9.jar:/root/hadoop/share/hadoop/common/lib/commons-httpclient-3.1.jar:/root/hadoop/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/root/hadoop/share/hadoop/common/lib/commons-collections-3.2.1.jar:/root/hadoop/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/root/hadoop/share/hadoop/common/lib/activation-1.1.jar:/root/hadoop/share/hadoop/common/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/root/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar:/root/hadoop/share/hadoop/common/lib/commons-digester-1.8.jar:/root/hadoop/share/hadoop/common/lib/servlet-api-2.5.jar:/root/hadoop/share/hadoop/common/lib/commons-lang-2.6.jar:/root/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/root/hadoop/share/hadoop/common/lib/zookeeper-3.4.5.jar:/root/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/root/hadoop/share/hadoop/common/hadoop-nfs-2.4.1.jar:/root/hadoop/share/hadoop/common/hadoop-common-2.4.1-tests.jar:/root/hadoop/share/hadoop/common/hadoop-common-2.4.1.jar:/root/hadoop/share/hadoop/hdfs:/root/hadoop/share/hadoop/hdfs/lib/asm-3.2.jar:/root/hadoop/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/root/hadoop/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/root/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/root/hadoop/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/root/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-el-1.0.jar:/root/hadoop/share/hadoop/hdfs/lib/guava-11.0.2.jar:/root/hadoop/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/root/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/root/hadoop/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/root/hadoop/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/root/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/root/hadoop/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/root/hadoop/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/root/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-2.4.1.jar:/root/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.4.1.jar:/root/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.4.1-tests.jar:/root/hadoop/share/hadoop/yarn/lib/asm-3.2.jar:/root/hadoop/share/hadoop/yarn/lib/xz-1.0.jar:/root/hadoop/share/hadoop/yarn/lib/jetty-6.1.26.jar:/root/hadoop/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/root/hadoop/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/root/hadoop/share/hadoop/yarn/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/yarn/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-xc-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/jsr305-1.3.9.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/commons-codec-1.4.jar:/root/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar:/root/hadoop/share/hadoop/yarn/lib/jline-0.9.94.jar:/root/hadoop/share/hadoop/yarn/lib/guava-11.0.2.jar:/root/hadoop/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/root/hadoop/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/root/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/root/hadoop/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-json-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/root/hadoop/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/root/hadoop/share/hadoop/yarn/lib/activation-1.1.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/root/hadoop/share/hadoop/yarn/lib/commons-cli-1.2.jar:/root/hadoop/share/hadoop/yarn/lib/servlet-api-2.5.jar:/root/hadoop/share/hadoop/yarn/lib/commons-lang-2.6.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-client-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/jettison-1.1.jar:/root/hadoop/share/hadoop/yarn/lib/zookeeper-3.4.5.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/guice-3.0.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-api-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-client-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/asm-3.2.jar:/root/hadoop/share/hadoop/mapreduce/lib/xz-1.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/hamcrest-core-1.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/root/hadoop/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/root/hadoop/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/root/hadoop/share/hadoop/mapreduce/lib/javax.inject-1.jar:/root/hadoop/share/hadoop/mapreduce/lib/hadoop-annotations-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/junit-4.10.jar:/root/hadoop/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/root/hadoop/share/hadoop/mapreduce/lib/guice-3.0.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.4.1-tests.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.4.1.jar:/root/hadoop/contrib/capacity-scheduler/*.jar
STARTUP_MSG:   build = http://svn.apache.org/repos/asf/hadoop/common -r 1604318; compiled by 'jenkins' on 2014-06-21T05:43Z
STARTUP_MSG:   java = 1.7.0_15
************************************************************/
15/06/19 19:28:45 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
15/06/19 19:28:45 INFO namenode.NameNode: createNameNode [-bootstrapStandby]
15/06/19 19:28:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/06/19 19:28:49 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/06/19 19:28:50 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/06/19 19:28:51 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/06/19 19:28:52 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/06/19 19:28:53 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/06/19 19:28:54 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/06/19 19:28:55 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/06/19 19:28:56 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/06/19 19:28:57 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/06/19 19:28:58 INFO ipc.Client: Retrying connect to server: v1/192.168.32.91:9000. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
15/06/19 19:28:58 FATAL ha.BootstrapStandby: Unable to fetch namespace information from active NN at v1/192.168.32.91:9000: Call From v2/192.168.32.92 to v1:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
15/06/19 19:28:58 INFO util.ExitUtil: Exiting with status 2
15/06/19 19:28:58 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at v2/192.168.32.92
************************************************************/

启动了active如下:

[root@v2 hadoop]# hdfs namenode -bootstrapStandby
15/06/19 19:31:39 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = v2/192.168.32.92
STARTUP_MSG:   args = [-bootstrapStandby]
STARTUP_MSG:   version = 2.4.1
STARTUP_MSG:   classpath = /root/hadoop/etc/hadoop:/root/hadoop/share/hadoop/common/lib/asm-3.2.jar:/root/hadoop/share/hadoop/common/lib/xz-1.0.jar:/root/hadoop/share/hadoop/common/lib/junit-4.8.2.jar:/root/hadoop/share/hadoop/common/lib/httpcore-4.2.5.jar:/root/hadoop/share/hadoop/common/lib/jetty-6.1.26.jar:/root/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/root/hadoop/share/hadoop/common/lib/jetty-util-6.1.26.jar:/root/hadoop/share/hadoop/common/lib/commons-compress-1.4.1.jar:/root/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/common/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/common/lib/hadoop-auth-2.4.1.jar:/root/hadoop/share/hadoop/common/lib/stax-api-1.0-2.jar:/root/hadoop/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/root/hadoop/share/hadoop/common/lib/netty-3.6.2.Final.jar:/root/hadoop/share/hadoop/common/lib/httpclient-4.2.5.jar:/root/hadoop/share/hadoop/common/lib/jsr305-1.3.9.jar:/root/hadoop/share/hadoop/common/lib/avro-1.7.4.jar:/root/hadoop/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/root/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/common/lib/commons-codec-1.4.jar:/root/hadoop/share/hadoop/common/lib/commons-el-1.0.jar:/root/hadoop/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/root/hadoop/share/hadoop/common/lib/guava-11.0.2.jar:/root/hadoop/share/hadoop/common/lib/xmlenc-0.52.jar:/root/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/root/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/root/hadoop/share/hadoop/common/lib/mockito-all-1.8.5.jar:/root/hadoop/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/root/hadoop/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/root/hadoop/share/hadoop/common/lib/jsch-0.1.42.jar:/root/hadoop/share/hadoop/common/lib/hadoop-annotations-2.4.1.jar:/root/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/root/hadoop/share/hadoop/common/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/common/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar:/root/hadoop/share/hadoop/common/lib/jets3t-0.9.0.jar:/root/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/root/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/root/hadoop/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/root/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/root/hadoop/share/hadoop/common/lib/jersey-json-1.9.jar:/root/hadoop/share/hadoop/common/lib/commons-httpclient-3.1.jar:/root/hadoop/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/root/hadoop/share/hadoop/common/lib/commons-collections-3.2.1.jar:/root/hadoop/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/root/hadoop/share/hadoop/common/lib/activation-1.1.jar:/root/hadoop/share/hadoop/common/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/root/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar:/root/hadoop/share/hadoop/common/lib/commons-digester-1.8.jar:/root/hadoop/share/hadoop/common/lib/servlet-api-2.5.jar:/root/hadoop/share/hadoop/common/lib/commons-lang-2.6.jar:/root/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/root/hadoop/share/hadoop/common/lib/zookeeper-3.4.5.jar:/root/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/root/hadoop/share/hadoop/common/hadoop-nfs-2.4.1.jar:/root/hadoop/share/hadoop/common/hadoop-common-2.4.1-tests.jar:/root/hadoop/share/hadoop/common/hadoop-common-2.4.1.jar:/root/hadoop/share/hadoop/hdfs:/root/hadoop/share/hadoop/hdfs/lib/asm-3.2.jar:/root/hadoop/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/root/hadoop/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/root/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/root/hadoop/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/root/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-el-1.0.jar:/root/hadoop/share/hadoop/hdfs/lib/guava-11.0.2.jar:/root/hadoop/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/root/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/root/hadoop/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/root/hadoop/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/root/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/root/hadoop/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/root/hadoop/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/root/hadoop/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/root/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-2.4.1.jar:/root/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.4.1.jar:/root/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.4.1-tests.jar:/root/hadoop/share/hadoop/yarn/lib/asm-3.2.jar:/root/hadoop/share/hadoop/yarn/lib/xz-1.0.jar:/root/hadoop/share/hadoop/yarn/lib/jetty-6.1.26.jar:/root/hadoop/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/root/hadoop/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/root/hadoop/share/hadoop/yarn/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/yarn/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-xc-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/jsr305-1.3.9.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/commons-codec-1.4.jar:/root/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar:/root/hadoop/share/hadoop/yarn/lib/jline-0.9.94.jar:/root/hadoop/share/hadoop/yarn/lib/guava-11.0.2.jar:/root/hadoop/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/root/hadoop/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/root/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/root/hadoop/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-json-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/root/hadoop/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/root/hadoop/share/hadoop/yarn/lib/activation-1.1.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/root/hadoop/share/hadoop/yarn/lib/commons-cli-1.2.jar:/root/hadoop/share/hadoop/yarn/lib/servlet-api-2.5.jar:/root/hadoop/share/hadoop/yarn/lib/commons-lang-2.6.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/jersey-client-1.9.jar:/root/hadoop/share/hadoop/yarn/lib/jettison-1.1.jar:/root/hadoop/share/hadoop/yarn/lib/zookeeper-3.4.5.jar:/root/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-1.8.8.jar:/root/hadoop/share/hadoop/yarn/lib/guice-3.0.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-api-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-client-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.4.1.jar:/root/hadoop/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/asm-3.2.jar:/root/hadoop/share/hadoop/mapreduce/lib/xz-1.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/hamcrest-core-1.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/root/hadoop/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/root/hadoop/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/root/hadoop/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/root/hadoop/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/root/hadoop/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/root/hadoop/share/hadoop/mapreduce/lib/javax.inject-1.jar:/root/hadoop/share/hadoop/mapreduce/lib/hadoop-annotations-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/root/hadoop/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/root/hadoop/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/root/hadoop/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/root/hadoop/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/root/hadoop/share/hadoop/mapreduce/lib/junit-4.10.jar:/root/hadoop/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/root/hadoop/share/hadoop/mapreduce/lib/guice-3.0.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.4.1-tests.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.4.1.jar:/root/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.4.1.jar:/root/hadoop/contrib/capacity-scheduler/*.jar
STARTUP_MSG:   build = http://svn.apache.org/repos/asf/hadoop/common -r 1604318; compiled by 'jenkins' on 2014-06-21T05:43Z
STARTUP_MSG:   java = 1.7.0_15
************************************************************/
15/06/19 19:31:39 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
15/06/19 19:31:39 INFO namenode.NameNode: createNameNode [-bootstrapStandby]
15/06/19 19:31:41 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
=====================================================
About to bootstrap Standby ID bnn2 from:
           Nameservice ID: bns1
        Other Namenode ID: bnn1
  Other NN's HTTP address: http://v1:50070
  Other NN's IPC  address: v1/192.168.32.91:9000
             Namespace ID: 32897138
            Block pool ID: BP-134496651-192.168.32.91-1434751114027
               Cluster ID: CID-c5890502-45a9-425e-9ae3-aa70c3eff8a6
           Layout version: -56
=====================================================
15/06/19 19:31:44 INFO common.Storage: Storage directory /root/hadoop/tmp/dfs/name has been successfully formatted.
15/06/19 19:31:45 INFO namenode.TransferFsImage: Opening connection to http://v1:50070/imagetransfer?getimage=1&txid=0&storageInfo=-56:32897138:0:CID-c5890502-45a9-425e-9ae3-aa70c3eff8a6
15/06/19 19:31:46 INFO namenode.TransferFsImage: Image Transfer timeout configured to 60000 milliseconds
15/06/19 19:31:46 INFO namenode.TransferFsImage: Transfer took 0.01s at 0.00 KB/s
15/06/19 19:31:46 INFO namenode.TransferFsImage: Downloaded file fsimage.ckpt_0000000000000000000 size 350 bytes.
15/06/19 19:31:46 INFO util.ExitUtil: Exiting with status 0
15/06/19 19:31:46 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at v2/192.168.32.92
************************************************************/


4.hbase启动时,如果不同节点时间不同的话,启动只有HMaster会起来,别的HRegionServer都会挂掉。同步时间即可,HBase需要用到hadoop的{core-site.xml/hdfs-site.xml},否则会将配置好的ha hdfsname(hdfs://bns1/hbase)当成hostname去解析,把hdfs相关文件拷贝到hbase/conf下就能正常运行了。
5.hive配置元数据快记法:
    4个配置项前缀:javax.jdo.option.Connection[URL/UserName/Password/DriverName]
    createDatabaseIfNotExist
    配置mysql允许远程连接:
        #(执行下面的语句  *.*:所有库下的所有表   %:任何IP地址或主机都可以连接)
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123' WITH GRANT OPTION;
FLUSH PRIVILEGES;
        如果这么配置完了之后,可以远程登录但是本机登录不了的时候。可以用mysqladmin给本地重新修改下密码
        /usr/bin/mysqladmin -uroot -h currentHostName password 'new-password'


0
0

查看评论
* 以上用户言论只代表其个人观点,不代表CSDN网站的观点或立场
    个人资料
    • 访问:66323次
    • 积分:1287
    • 等级:
    • 排名:千里之外
    • 原创:58篇
    • 转载:21篇
    • 译文:0篇
    • 评论:3条
    博客搬家啦 ~
    文章分类