HBase 0.94.21 zookeeper-3.4.6 分布式安装

1：配置hosts和ssh无密码登陆

2：配置hbase

解压tar zxf hbase-0.94.21.tar.gz

配置hbase-env.sh

[jifeng@jifeng02 hbase-0.94.21]$ cd conf
[jifeng@jifeng02 conf]$ cat hbase-env.sh 
#
#/**
# * Copyright 2007 The Apache Software Foundation
# *
# * Licensed to the Apache Software Foundation (ASF) under one
# * or more contributor license agreements.  See the NOTICE file
# * distributed with this work for additional information
# * regarding copyright ownership.  The ASF licenses this file
# * to you under the Apache License, Version 2.0 (the
# * "License"); you may not use this file except in compliance
# * with the License.  You may obtain a copy of the License at
# *
# *     http://www.apache.org/licenses/LICENSE-2.0
# *
# * Unless required by applicable law or agreed to in writing, software
# * distributed under the License is distributed on an "AS IS" BASIS,
# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# * See the License for the specific language governing permissions and
# * limitations under the License.
# */

# Set environment variables here.

# This script sets variables multiple times over the course of starting an hbase process,
# so try to keep things idempotent unless you want to take an even deeper look
# into the startup scripts (bin/hbase, etc.)

# The java implementation to use.  Java 1.6 required.
export JAVA_HOME=$HOME/jdk1.7.0_45

# Extra Java CLASSPATH elements.  Optional.
export HBASE_CLASSPATH=$HOME/hadoop/hadoop-1.2.1/conf

# The maximum amount of heap to use, in MB. Default is 1000.
# export HBASE_HEAPSIZE=1000

# Extra Java runtime options.
# Below are what we set by default.  May only work with SUN JVM.
# For more on why as well as other possible settings,
# see http://wiki.apache.org/hadoop/PerformanceTuning
export HBASE_OPTS="-XX:+UseConcMarkSweepGC"

# Uncomment one of the below three options to enable java garbage collection logging for the server-side processes.

# This enables basic gc logging to the .out file.
# export SERVER_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps"

# This enables basic gc logging to its own file.
# If FILE-PATH is not replaced, the log file(.gc) would still be generated in the HBASE_LOG_DIR .
# export SERVER_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:<FILE-PATH>"

# This enables basic GC logging to its own file with automatic log rolling. Only applies to jdk 1.6.0_34+ and 1.7.0_2+.
# If FILE-PATH is not replaced, the log file(.gc) would still be generated in the HBASE_LOG_DIR .
# export SERVER_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:<FILE-PATH> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=1 -XX:GCLogFileSize=512M"

# Uncomment one of the below three options to enable java garbage collection logging for the client processes.

# This enables basic gc logging to the .out file.
# export CLIENT_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps"

# This enables basic gc logging to its own file.
# If FILE-PATH is not replaced, the log file(.gc) would still be generated in the HBASE_LOG_DIR .
# export CLIENT_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:<FILE-PATH>"

# This enables basic GC logging to its own file with automatic log rolling. Only applies to jdk 1.6.0_34+ and 1.7.0_2+.
# If FILE-PATH is not replaced, the log file(.gc) would still be generated in the HBASE_LOG_DIR .
# export CLIENT_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:<FILE-PATH> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=1 -XX:GCLogFileSize=512M"

# Uncomment below if you intend to use the EXPERIMENTAL off heap cache.
# export HBASE_OPTS="$HBASE_OPTS -XX:MaxDirectMemorySize="
# Set hbase.offheapcache.percentage in hbase-site.xml to a nonzero value.


# Uncomment and adjust to enable JMX exporting
# See jmxremote.password and jmxremote.access in $JRE_HOME/lib/management to configure remote password access.
# More details at: http://java.sun.com/javase/6/docs/technotes/guides/management/agent.html
#
# export HBASE_JMX_BASE="-Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false"
# export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10101"
# export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10102"
# export HBASE_THRIFT_OPTS="$HBASE_THRIFT_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10103"
# export HBASE_ZOOKEEPER_OPTS="$HBASE_ZOOKEEPER_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10104"

# File naming hosts on which HRegionServers will run.  $HBASE_HOME/conf/regionservers by default.
# export HBASE_REGIONSERVERS=${HBASE_HOME}/conf/regionservers

# File naming hosts on which backup HMaster will run.  $HBASE_HOME/conf/backup-masters by default.
# export HBASE_BACKUP_MASTERS=${HBASE_HOME}/conf/backup-masters

# Extra ssh options.  Empty by default.
# export HBASE_SSH_OPTS="-o ConnectTimeout=1 -o SendEnv=HBASE_CONF_DIR"

# Where log files are stored.  $HBASE_HOME/logs by default.
# export HBASE_LOG_DIR=${HBASE_HOME}/logs

# Enable remote JDWP debugging of major HBase processes. Meant for Core Developers 
# export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8070"
# export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8071"
# export HBASE_THRIFT_OPTS="$HBASE_THRIFT_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8072"
# export HBASE_ZOOKEEPER_OPTS="$HBASE_ZOOKEEPER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8073"

# A string representing this instance of hbase. $USER by default.
# export HBASE_IDENT_STRING=$USER

# The scheduling priority for daemon processes.  See 'man nice'.
# export HBASE_NICENESS=10

# The directory where pid files are stored. /tmp by default.
export HBASE_PID_DIR=/home/jifeng/pids

# Seconds to sleep between slave commands.  Unset by default.  This
# can be useful in large clusters, where, e.g., slave rsyncs can
# otherwise arrive faster than the master can service them.
# export HBASE_SLAVE_SLEEP=0.1

# Tell HBase whether it should manage it's own instance of Zookeeper or not.
export HBASE_MANAGES_ZK=true

修改的地方
#JDK的安装目录
export JAVA_HOME=$HOME/jdk1.7.0_45
#hadoop的配置目录
export HBASE_CLASSPATH=$HOME/hadoop/hadoop-1.2.1/conf

和最后一行

export HBASE_MANAGES_ZK=true

#true：表示zookeeper交给hbase管理，启动hbase时，会自动启动hbase-site.xml里的hbase.zookeeper.quorum属性中的所有zookeeper实例 #false：表示启动hbase时，要手动启动所有zookeeper实例

配置hbase-site.xml

[jifeng@jifeng02 conf]$ cat hbase-site.xml 
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
/**
 * Copyright 2010 The Apache Software Foundation
 *
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership.  The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
-->
<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://jifeng01:9000/user/jifeng/hbase</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
   <name>hbase.master</name>
   <value>jifeng01:60000</value>
 </property>
 <property>
 <name>hbase.zookeeper.quorum</name>
  <value>jifeng01,jifeng02</value>
 </property>

</configuration>

配置regionservers 

[jifeng@jifeng02 conf]$ cat regionservers 
jifeng01
jifeng02

3：配置zookeeper

解压 tar zxf zookeeper-3.4.6.tar.gz

[jifeng@jifeng02 conf]# cp zoo_sample.cfg zoo.cfg
[jifeng@jifeng02 conf]$ vi zoo.cfg
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/home/jifeng/zookeeper-3.4.6/data
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server.1=jifeng01:2888:3888
server.2=jifeng02:2888:3888

4: hadoop-core-1.2.1.jar替换hbase-0.94.21/lib目录下的hadoop-core-1.0.4.jar

[jifeng@jifeng02 hadoop-1.2.1]$ cp ./hadoop-core-1.2.1.jar ../../hbase-0.94.21/lib
[jifeng@jifeng02 hadoop-1.2.1]$ rm hadoop-core-1.0.4.jar

5:配置环境变量和复制文件到jifeng01

[jifeng@jifeng01 ~]$ cat .bash_profile
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin

export PATH
export JAVA_HOME=$HOME/jdk1.7.0_45
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export HADOOP_HOME=$HOME/hadoop/hadoop-1.2.1
export ANT_HOME=$HOME/apache-ant-1.9.4
export HBASE_HOME=$HOME/hbase-0.94.21

expor PATH=$PATH:$ANT_HOME/bin:$HBASE_HOME/bin

scp -r hbase-0.94.21 jifeng@jifeng01:/home/jifeng

scp -r zookeeper-3.4.6 jifeng@jifeng01:/home/jifeng

6:启动HBase

[jifeng@jifeng01 ~]$ cd hbase-0.94.21
[jifeng@jifeng01 hbase-0.94.21]$ bin/start-hbase.sh
localhost: starting zookeeper, logging to /home/jifeng/hbase-0.94.21/bin/../logs/hbase-jifeng-zookeeper-jifeng01.out
starting master, logging to /home/jifeng/hbase-0.94.21/logs/hbase-jifeng-master-jifeng01.out
jifeng01: starting regionserver, logging to /home/jifeng/hbase-0.94.21/bin/../logs/hbase-jifeng-regionserver-jifeng01.out
jifeng02: starting regionserver, logging to /home/jifeng/hbase-0.94.21/bin/../logs/hbase-jifeng-regionserver-jifeng02.out
[jifeng@jifeng01 hbase-0.94.21]$ jps
15627 HQuorumPeer
15705 HMaster
15880 HRegionServer
4529 NameNode
16040 Jps
4822 JobTracker
[jifeng@jifeng01 hbase-0.94.21]$ bin/hbase shell
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 0.94.21, r83b4a1ee9b9a2fa4c7ae1739259e041cabe8edc2, Fri Jun 27 16:14:16 UTC 2014

hbase(main):001:0> status
1 servers, 0 dead, 2.0000 average load

hbase(main):002:0> exit

只启动了一台服务器

7：查看日志

[jifeng@jifeng02 logs]$ cat hbase-jifeng-regionserver-jifeng02.log

2014-08-16 15:37:57,847 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Telling master at jifeng01,60000,1408174832154 that we are up with port=60020, startcode=1408174674748
2014-08-16 15:37:57,854 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: Master rejected startup because clock is out of sync
org.apache.hadoop.hbase.ClockOutOfSyncException: org.apache.hadoop.hbase.ClockOutOfSyncException: Server jifeng02,60020,1408174674748 has been rejected; Reported time is too far out of sync with master.  Time difference of 158488ms > max allowed of 30000ms
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
        at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2116)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:753)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase.ClockOutOfSyncException: Server jifeng02,60020,1408174674748 has been rejected; Reported time is too far out of sync with master.  Time difference of 158488ms > max allowed of 30000ms
        at org.apache.hadoop.hbase.master.ServerManager.checkClockSkew(ServerManager.java:235)
        at org.apache.hadoop.hbase.master.ServerManager.regionServerStartup(ServerManager.java:169)
        at org.apache.hadoop.hbase.master.HMaster.regionServerStartup(HMaster.java:1084)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:323)
        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1434)


        at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1012)
        at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:87)
        at com.sun.proxy.$Proxy9.regionServerStartup(Unknown Source)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2114)
        ... 2 more

时间不同步的问题，

解决方法：

[root@jifeng02 ~]# date -s "2015-08-16 16:10:00"; clock -w

8：重新启动hbase

停止[jifeng@jifeng01 hbase-0.94.21]$ bin/stop-hbase.sh

如果jifeng02的服务没全部关闭，先用

[jifeng@jifeng02 ~]$ jps
19487 HQuorumPeer
19577 HRegionServer
19839 Jps
3982 TaskTracker
3888 DataNode
[jifeng@jifeng02 ~]$ 

然后 kill -s 9 19487
然后 kill -s 9 19577

[jifeng@jifeng01 hbase-0.94.21]$ bin/hbase shell
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 0.94.21, r83b4a1ee9b9a2fa4c7ae1739259e041cabe8edc2, Fri Jun 27 16:14:16 UTC 2014


hbase(main):001:0> status
2 servers, 0 dead, 1.0000 average load


hbase(main):002:0>