HBase集群搭建1Master、2Slaves

HBase集群

Master.Hadoop 10.9.148.21 master(root,123456) /softs/hbase  /softs/ hbase/logs

Slave1.Hadoop 10.9.148.26  slave1(root,123456) /softs/hbase  /softs/ hbase/logs

Slave2.Hadoop 10.9.148.23  slave2(root,123456) /softs/hbase  /softs/hbase/logs

 

hbase-1.2.6-bin.tar.gz

 826  scp hbase-1.2.6-bin.tar.gzroot@10.9.148.26:/softs/

 827  scp hbase-1.2.6-bin.tar.gzroot@10.9.148.23:/softs/

Master slave 相同：

tar –zxf hbase-1.2.6-bin.tar.gz

 830  mv hbase-1.2.6 hbase

 833  cd hbase

 836  vi /etc/profile

添加内容：

#hbase     

exportHBASE_HOME=/softs/hbase

exportPATH=$PATH:$HBASE_HOME

#stop hbase zookeeperrunning

exportHBASE_MANAGES_ZK=false

      vi /etc/hosts

#zookeeper MasterSlave IP address

10.9.148.21 master

10.9.148.26 slave1

10.9.148.23 slave2

 837  source /etc/profile

 839  vi conf/hbase-site.xml

添加内容：

<configuration>

        <property>

               <name>hbase.rootdir</name>

               <value>hdfs://10.9.148.21:9000/hbase</value>

        </property>

        <property>

               <name>hbase.cluster.distributed</name>

                <value>true</value>

        </property>

        <property>

               <name>hbase.master</name>

                <value>master</value>

        </property>

        <property>

               <name>hbase.zookeeper.quorum</name>

               <value>master:2181,slave1:2181,slave2:2181</value>

        </property>

        <property>

               <name>hbase.zookeeper.property.dataDir</name>

               <value>/softs/zookeeper/data</value>

        </property>

</configuration>

 840  vi conf/regionservers

添加内容：

slave1

slave2

     vi conf/hbase-env.sh

添加内容：

export JAVA_HOME=/usr/local/jdk1.7

export HBASE_MANAGES_ZK=false

 

启动HBase集群

启动之前，在Hadoop的文件系统中创建hbase目录

  850 start-all.sh

查看./logs目录下的

vihbase/logs/hbase-root-master-Master.Hadoop.log

发现slave1和slave2没有启动，错误提示：连接master超时（或者hbase集群节点HRegionServer启动后自动关闭）

出现问题的原因：3台主机的系统时钟不一致，相差超过30s导致。

解决问题：使用工具，统一时间（集群时钟管理工具）

           在这里，我是用的是直接设置时钟：

           1052 date -s 11:41:00

 1053 clock -w

重新启动集群

1057  bin/start-hbase.sh

验证：

Master：

[root@Master softs]# jps

3717 SecondaryNameNode

4606 QuorumPeerMain

13543 Jps

6909 HMaster

6756 NameNode

3911 ResourceManager

[root@Master softs]#

Slave1：

[root@Slave1 hbase]# jps

24344 Jps

4148 SecondaryNameNode

3058 QuorumPeerMain

3981 DataNode

4310 ResourceManager

4427 NodeManager

23638 HRegionServer

[root@Slave1 hbase]#

Slave2：

[root@Slave2 hbase]# jps

3596 QuorumPeerMain

4048 DataNode

11495 HRegionServer

12157 Jps

[root@Slave2 hbase]#

在浏览器访问主节点16010端口（即http://10.9.148.21:16010/），可以产看HBase Web UI

在hadoop的文件系统中：

HBase启动几次失败，后来改正HBase配置后，依旧失败。可以试着将hadoop的HBase的文件目录删掉重建，再重新启动HBase集群。这样可以成功。

HBaseJava使用

 

 

Maven依赖：

<!-- hbase -->

  <dependency>

   <groupId>org.apache.hbase</groupId>

   <artifactId>hbase-it</artifactId>

   <version>1.2.6</version>

  </dependency>

  <!--spring -->

            <dependency>

            <groupId>org.springframework</groupId>

            <artifactId>spring-context-support</artifactId>

            <version>3.2.13.RELEASE</version>

       </dependency>

本地计算机C:\WINDOWS\system32\drivers\etc\hosts

增加：

10.9.148.21 master

10.9.148.26 slave1

10.9.148.23 slave2

为HBase Java API的配置文件做准备：

   static{

      conf=HBaseConfiguration.create();

        conf.set("hbase.rootdir","hdfs://10.9.148.21:9000/hbase");

        conf.set("hbase.master","10.9.148.21");

        conf.set("hbase.zookeeper.quorum","10.9.148.21,10.9.148.26,10.9.148.23");

         }

Java操作HBase集群，可以创建表删除表。当进行对表的put、get、admin.tableExists(tablename)等时，运行java代码出错。查看zookeeper日志发现：

2017-08-03 18:42:58,643 [myid:1] - INFO  [QuorumPeer[myid=1]/0.0.0.0:2181:Learner@326]- Getting a diff from the leader 0x1000000a7

2017-08-03 18:42:58,644 [myid:1] - INFO [QuorumPeer[myid=1]/0.0.0.0:2181:FileTxnSnapLog@240] - Snapshotting:0x1000000a7 to ./data/version-2/snapshot.1000000a7

2017-08-03 18:43:00,905 [myid:1] - INFO  [/10.9.148.21:3888:QuorumCnxManager$Listener@541]- Received connection request /10.9.148.23:43171

2017-08-03 18:43:00,908 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1(message format version), 3 (n.leader), 0x1000000a7 (n.zxid), 0x1 (n.round),LOOKING (n.state), 3 (n.sid), 0x1 (n.peerEpoch) FOLLOWING (my state)

2017-08-03 18:43:00,922 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1(message format version), 3 (n.leader), 0x1000000a7 (n.zxid), 0x2 (n.round),LOOKING (n.state), 3 (n.sid), 0x1 (n.peerEpoch) FOLLOWING (my state)

2017-08-03 18:43:23,050 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] -Accepted socket connection from /10.9.148.21:20942

2017-08-03 18:43:23,063 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@900] - Clientattempting to establish new session at /10.9.148.21:20942

2017-08-03 18:43:23,065 [myid:1] - WARN [QuorumPeer[myid=1]/0.0.0.0:2181:Follower@118] - Got zxid 0x200000001expected 0x1

2017-08-03 18:43:23,071 [myid:1] - INFO  [CommitProcessor:1:ZooKeeperServer@645] -Established session 0x15da7b0f5d30000 with negotiated timeout 800000 for client/10.9.148.21:20942

2017-08-03 18:43:24,321 [myid:1] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192]- Accepted socket connection from /10.9.148.23:37715

2017-08-03 18:43:24,323 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@900] - Clientattempting to establish new session at /10.9.148.23:37715

2017-08-03 18:43:24,326 [myid:1] - INFO  [CommitProcessor:1:ZooKeeperServer@645] -Established session 0x15da7b0f5d30001 with negotiatedtimeout 800000 for client /10.9.148.23:37715

2017-08-03 18:43:25,495 [myid:1] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192]- Accepted socket connection from /10.9.148.21:20951

2017-08-03 18:43:25,496 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@900] - Clientattempting to establish new session at /10.9.148.21:20951

2017-08-03 18:43:25,499 [myid:1] - INFO  [CommitProcessor:1:ZooKeeperServer@645] -Established session 0x15da7b0f5d30002 with negotiated timeout 800000 for client/10.9.148.21:20951

Zoomkeeper连接超时。

解决方法：

在运行java代码的windows本地计算机中：C:\WINDOWS\system32\drivers\etc\hosts

增加：

10.9.148.21 Master.Hadoop

10.9.148.26 Slave1.Hadoop

10.9.148.23 Slave2.Hadoop

Master.Hadoop Slave1.Hadoop Slave2.Hadoop分别为三台服务器的主机名。

如此操作成功：

常用操作：

 

对表的操作

创建表