段智华的博客

热烈祝贺王家林大咖2018年清华大学出版新书《SPARK大数据商业实战三部曲》!清华大学出版社官方旗舰店(天猫) https://qhdx.tmall.com/?spm=a220o.1000855.1...

HBase实战(2):搭建分布式集群HBASE

搭建分布式集群HBASE

1,单机版本的安装请参考 https://blog.csdn.net/duan_zhihua/article/details/80622166

2,ZOOKEPER集群的安装请参考https://blog.csdn.net/duan_zhihua/article/details/70885469

2,分布式集群HBASE安装: 

  • 配置vim hbase-env.sh  
root@master:/usr/local/hbase-1.2.0/conf# vi hbase-env.sh
export JAVA_HOME=/usr/local/jdk1.8.0_60
export HBASE_MANAGES_ZK=false 
  • 配置vim hbase-site.xml 
  •  <configuration>  
         
            <property>  
                <name>hbase.rootdir</name>  
                <value>hdfs://master:9000/hbase</value>  
            </property>  
            
            <property>  
                <name>hbase.cluster.distributed</name>  
                <value>true</value>  
            </property>  
              
            <property>  
                <name>hbase.zookeeper.quorum</name>  
                <value>192.168.189.1:2181,192.168.189.2:2181,192.168.189.3:2181</value>  
            </property>  
        </configuration>  
    配置regionservers
  • root@master:/usr/local/hbase-1.2.0/conf# vi regionservers
    
    master
    worker1
    worker2
    worker3
    ~
    ~
    从Master复制hbase目录到从节点worker1 worker2 worker3中
  • scp -r /usr/local/hbase-1.2.0 worker1:/usr/local/hbase-1.2.0
    scp -r /usr/local/hbase-1.2.0 worker2:/usr/local/hbase-1.2.0
    scp -r /usr/local/hbase-1.2.0 worker3:/usr/local/hbase-1.2.0
    依次启动Hadoop、zookeeper、hbase
  • 启动hadoop
  • root@master:~# /usr/local/hadoop-2.6.0/sbin/start-all.sh
    This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:file:/usr/local/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/usr/local/alluxio-1.7.0-hadoop-2.6/client/alluxio-1.7.0-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
    Starting namenodes on [master]
    master: starting namenode, logging to /usr/local/hadoop-2.6.0/logs/hadoop-root-namenode-master.out
    worker3: starting datanode, logging to /usr/local/hadoop-2.6.0/logs/hadoop-root-datanode-worker3.out
    worker1: starting datanode, logging to /usr/local/hadoop-2.6.0/logs/hadoop-root-datanode-worker1.out
    worker2: starting datanode, logging to /usr/local/hadoop-2.6.0/logs/hadoop-root-datanode-worker2.out
    Starting secondary namenodes [0.0.0.0]
    0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop-2.6.0/logs/hadoop-root-secondarynamenode-master.out
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:file:/usr/local/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/usr/local/alluxio-1.7.0-hadoop-2.6/client/alluxio-1.7.0-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
    starting yarn daemons
    starting resourcemanager, logging to /usr/local/hadoop-2.6.0/logs/yarn-root-resourcemanager-master.out
    worker2: starting nodemanager, logging to /usr/local/hadoop-2.6.0/logs/yarn-root-nodemanager-worker2.out
    worker1: starting nodemanager, logging to /usr/local/hadoop-2.6.0/logs/yarn-root-nodemanager-worker1.out
    worker3: starting nodemanager, logging to /usr/local/hadoop-2.6.0/logs/yarn-root-nodemanager-worker3.out
    root@master:~# hadoop dfsadmin -safemode leave 
    DEPRECATED: Use of this script to execute hdfs command is deprecated.
    Instead use the hdfs command for it.
    
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:file:/usr/local/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/usr/local/alluxio-1.7.0-hadoop-2.6/client/alluxio-1.7.0-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
    Safe mode is OFF
    root@master:~# 
root@master:~# jps
2722 SecondaryNameNode
3204 Jps
2893 ResourceManager
2542 NameNode
root@master:~# 
启动zookeeper

root@master:~# zkServer.sh start 
JMX enabled by default
Using config: /usr/local/zookeeper-3.4.6/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
root@master:~# zkServer.sh  status  
JMX enabled by default
Using config: /usr/local/zookeeper-3.4.6/bin/../conf/zoo.cfg
Mode: follower
root@master:~# jps
2722 SecondaryNameNode
3385 Jps
3324 QuorumPeerMain
2542 NameNode
root@master:~# 
root@worker1:/usr/local# zkServer.sh start 
JMX enabled by default
Using config: /usr/local/zookeeper-3.4.6/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
root@worker1:/usr/local# zkServer.sh  status  
JMX enabled by default
Using config: /usr/local/zookeeper-3.4.6/bin/../conf/zoo.cfg
Mode: leader
root@worker1:/usr/local# jps
2800 Jps
2530 NodeManager
2741 QuorumPeerMain
2399 DataNode
root@worker2:~# zkServer.sh start 
JMX enabled by default
Using config: /usr/local/zookeeper-3.4.6/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
root@worker2:~# zkServer.sh  status  
JMX enabled by default
Using config: /usr/local/zookeeper-3.4.6/bin/../conf/zoo.cfg
Mode: follower
root@worker2:~# jps
2707 Jps
2654 QuorumPeerMain
2287 DataNode

启动HBASE
root@master:~# start-hbase.sh
starting master, logging to /usr/local/hbase-1.2.0/logs/hbase-root-master-master.out
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
worker1: starting regionserver, logging to /usr/local/hbase-1.2.0/bin/../logs/hbase-root-regionserver-worker1.out
worker2: starting regionserver, logging to /usr/local/hbase-1.2.0/bin/../logs/hbase-root-regionserver-worker2.out
worker3: starting regionserver, logging to /usr/local/hbase-1.2.0/bin/../logs/hbase-root-regionserver-worker3.out
master: starting regionserver, logging to /usr/local/hbase-1.2.0/bin/../logs/hbase-root-regionserver-master.out
worker2: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
worker2: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
worker1: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
worker1: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
worker3: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
worker3: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
master: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
master: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
root@master:~# jps
4002 Jps
2722 SecondaryNameNode
3687 HRegionServer
3324 QuorumPeerMain
3532 HMaster
2542 NameNode
root@master:~# 
root@worker1:/usr/local# jps
2530 NodeManager
2741 QuorumPeerMain
3035 Jps
2878 HRegionServer
2399 DataNode
root@worker1:/usr/local# 
root@worker2:~# jps
2707 Jps
2654 QuorumPeerMain
2287 DataNode
root@worker2:~# jps
2792 HRegionServer
3005 Jps
2654 QuorumPeerMain
2287 DataNode
root@worker2:~# 

root@worker3:~# jps
2711 HRegionServer
2344 DataNode
2921 Jps
2479 NodeManager
root@worker3:~# 




在hbase-site.xml中加入web的配置:

<property>
        <name>hbase.master.info.port</name>
        <value>60010</value>
</property>

hbase-site.xml复制到worker节点

root@master:/usr/local/hbase-1.2.0/conf# scp -r /usr/local/hbase-1.2.0/conf/hbase-site.xml worker1:/usr/local/hbase-1.2.0/conf/hbase-site.xml
hbase-site.xml                                                                                                                        100%  603     0.6KB/s   00:00    
root@master:/usr/local/hbase-1.2.0/conf# scp -r /usr/local/hbase-1.2.0/conf/hbase-site.xml worker2:/usr/local/hbase-1.2.0/conf/hbase-site.xml
hbase-site.xml                                                                                                                        100%  603     0.6KB/s   00:00    
root@master:/usr/local/hbase-1.2.0/conf# scp -r /usr/local/hbase-1.2.0/conf/hbase-site.xml worker3:/usr/local/hbase-1.2.0/conf/hbase-site.xml
hbase-site.xml                                                                                                                        100%  603     0.6KB/s   00:00    
root@master:/usr/local/hbase-1.2.0/conf# 

重启hbase。

root@master:~# start-hbase.sh
starting master, logging to /usr/local/hbase-1.2.0/logs/hbase-root-master-master.out
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
worker1: starting regionserver, logging to /usr/local/hbase-1.2.0/bin/../logs/hbase-root-regionserver-worker1.out
master: starting regionserver, logging to /usr/local/hbase-1.2.0/bin/../logs/hbase-root-regionserver-master.out
worker3: starting regionserver, logging to /usr/local/hbase-1.2.0/bin/../logs/hbase-root-regionserver-worker3.out
worker2: starting regionserver, logging to /usr/local/hbase-1.2.0/bin/../logs/hbase-root-regionserver-worker2.out
worker1: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
worker1: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
master: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
master: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
worker3: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
worker3: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
worker2: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
worker2: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
root@master:~# 

打开HBASE的web页面地址http://192.168.189.1:60010/master-status,部署成功!



进入hbase shell

root@master:~# hbase shell
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hbase-1.2.0/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/alluxio-1.7.0-hadoop-2.6/client/alluxio-1.7.0-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 1.2.0, r25b281972df2f5b15c426c8963cbf77dd853a5ad, Thu Feb 18 23:01:49 CST 2016

hbase(main):001:0> 

在hbase中分布式集群中操作:


在web页面检查新建的表


阅读更多

扫码向博主提问

去开通我的Chat快问

duan_zhihua

博客专家

非学,无以致疑;非问,无以广识
  • 擅长领域:
  • 王家林老师AI盘古
  • AI人工智能作业答疑
  •  Spark+AI
版权声明:王家林大咖2018年新书《SPARK大数据商业实战三部曲》清华大学出版,清华大学出版社官方旗舰店(天猫)https://qhdx.tmall.com/?spm=a220o.1000855.1997427721.d4918089.4b2a2e5dT6bUsM https://blog.csdn.net/duan_zhihua/article/details/80664859
想对作者说点什么? 我来说一句

java大数据案例_3HBase

2018年01月23日 26KB 下载

没有更多推荐了,返回首页

加入CSDN,享受更精准的内容推荐,与500万程序员共同成长!
关闭
关闭