Hadoop生态系统搭建(4)——高性能分布式 NoSQL 数据库 HBase 的安装部署与测试

1. 系统环境

Ubuntu 16.04
vmware
Hadoop 2.7.0
Java 1.8.0_111

master:192.168.19.128
slave1:192.168.19.129
slave2:192.168.19.130

2. 安装部署

关键点:

  • Introduction to HBase Architecture
  • An In-Depth Look at the HBase Architecture
  • Region server manages regions and runs on HDFS DataNodes.
  • Master server assigns region to region server with the help of Apache Zookeeper
  • HBase uses Zookeeper as a coordinator service to maintain the server state in the cluster.
  • HBase takes care of the network partitions and client communicate with regions through Zookeeper.
  • Column families are stored in separate files in HDFS, which can be accessed separately.
  • Region servers serve data for reads and writes.
  • When accessing data, clients communicate with HBase RegionServers directly.
  • Region assignment, DDL (create, delete tables) operations are handled by the HBase Master process.
  • Region Servers are collocated with the HDFS DataNodes, which enable data locality (putting the data close to where it is needed) for the data served by the RegionServers.
  • A distributed Apache HBase installation depends on a running ZooKeeper cluster.


所以:NameNode 安装部署的组件为 HMaster,DataNode 安装部署的组件是 RegionServer,Zookeeper使用单独的节点(非NameNode非DataNode)进行部署。

1.各节点下载Hbase。

wget http://apache.fayea.com/hbase/1.2.5/hbase-1.2.5-bin.tar.gz
tar -zxf hbase-1.2.5-bin.tar.gz

2.各节点配置 HBASE_HOME 和 PATH。

# config hbase env
export HBASE_HOME=/home/hadoop/software/hbase-1.2.5
export PATH=$HBASE_HOME/bin:$PATH

3.各节点 conf 目录下配置hbase-env.sh,设置 。

# The java implementation to use.  Java 1.7+ required.
export JAVA_HOME=/usr/local/jdk1.8.0_111

# use special ZooKeeper,don't use HBase's own ZooKeeper
# HBASE_MANAGES_ZK 环境变量用来设置是使用hbase默认自带的 Zookeeper还是使用独立的ZooKeeper。HBASE_MANAGES_ZK=false 时使用独立的,为true时使用默认自带的
export HBASE_MANAGES_ZK=false

# The directory where pid files are stored. /tmp by default.Do not use /tmp (recommand),give directory's permission to hbase user.
export HBASE_PID_DIR=/var/hadoop/pids

Using HBase’s own Zookeeper

注意:使用自带的ZooKeeper时,需要将各节点的hbase-env.sh设置HBASE_MANAGES_ZK=true

4.1 在 HMaster 节点的 conf/hbase-site.xml配置如下:

<configuration>
  <property>
    <name>hbase.rootdir</name>
    <value>hdfs://master:9000/hbase</value>
  </property>

  <property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
  </property>

  <!-- use hbase's zookeeper -->
  <property>
      <name>hbase.zookeeper.quorum</name>
      <value>master,slave1,slave2</value>
  </property>

  <property>
    <name>hbase.zookeeper.property.dataDir</name>
    <value>hdfs://master:9000/zookeeper</value>
  </property>

  <property>
    <name>hbase.zookeeper.property.clientPort</name>
    <value>2181</value>
  </property>

  <property>
      <name>hbase.zookeeper.quorum</name>
      <value>master,slave1,slave2</value>
  </property>
  <!-- end use hbase's zookeeper -->

</configuration>

4.2 在Slave节点配置 conf/hbase-site.xml

<configuration>

  <property>
    <name>hbase.rootdir</name>
    <value>hdfs://master:9000/hbase</value>
  </property>
  <property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
  </property>

</configuration>

4.3 各节点配置conf/regionservers,a list of each node that should run a RegionServer in the cluster,These nodes need HBase installed and they need to use the same contents of the conf directory as the Master server.

slave1
slave2

4.4 start-hbase.sh启动 Hbase。
ZooKeeper starts first, followed by the master, then the RegionServers, and finally the backup masters.
这里写图片描述

Using existing ZooKeeper

4.1 set HBASE_MANAGES_ZK in conf/hbase-env.sh to false

export HBASE_MANAGES_ZK=false

4.3 各节点配置conf/regionservers,a list of each node that should run a RegionServer in the cluster,These nodes need HBase installed and they need to use the same contents of the conf directory as the Master server.

slave1
slave2

4.4 各节点conf/目录下backup-masters文件,假设将slave1节点作为 Master 的 backup。

slave1

4.5 配置 hbase-site.xml

<configuration>
  <property>
    <name>hbase.rootdir</name>
    <value>hdfs://master:9000/hbase</value>
  </property>

  <property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
  </property>

  <property>
      <name>hbase.zookeeper.quorum</name>
      <value>master,slave1,slave2</value>
  </property>

  <property>
    <name>hbase.zookeeper.property.dataDir</name>
    <!-- same as $$ZOOKEEPER_HOME/conf/zoo.cfg : dataDir=/home/hadoop/software/zookeeper-3.4.6/zookeeper-data -->
    <value>/home/hadoop/software/hbase-1.2.5/zookeeper-data</value>
  </property>

  <property>
    <name>hbase.zookeeper.property.clientPort</name>
    <value>2181</value>
  </property>
</configuration>

4.5 各节点zkServer.sh start先启动 zookeeper。
ZooKeeper starts first.




4.6 start-hbase.sh启动 Hbase。
master starts first, then the RegionServers, and finally the backup masters.



4.6 Web UI. http://master:16010




Done!

发布了72 篇原创文章 · 获赞 26 · 访问量 31万+
展开阅读全文

没有更多推荐了,返回首页

©️2019 CSDN 皮肤主题: 编程工作室 设计师: CSDN官方博客

分享到微信朋友圈

×

扫一扫,手机浏览