Hbase体系架构与安装

是什么

Hbase是一个非关系型的数据库

体系架构

client: 客户端

zookeeper

• 监控hmaster的状态,保证有且仅有一个活跃的hmaster,达到高可用.
• 存储meta表的寻址入口
• 感知regionserver的上下线信息,通知hmaster

Hmaster

• 为regionserver分配region
• 复杂regionserver的负载聚合

regionserver

• 管理本机的region
• 切分运行时变大的region

Hlog

记录hbase的操作,优先写入Hlog

region

一个表的一部分.

安装

以分布式安装为例

[root@mypc01 ~]# tar -zxvf hbase-1.2.1-bin.tar.gz -C /usr/local/
[root@mypc01 ~]# cd /usr/local/
[root@mypc01 local]# mv hbase-1.2.1 hbase
[root@mypc01 local]# vi /etc/profile
.......省略.......
#hbase environment
export HBASE_HOME=/usr/local/hbase
export PATH=$HBASE_HOME/bin:$PATH

[root@mypc01 local]# source  /etc/profile
[root@mypc01 local]# hbase version

禁用hbase自带zookeeper,用自己搭建的,需要修改hbase-env.sh文件

[root@mypc01 local]# vi $HBASE_HOME/conf/hbase-env.sh

#找到下面内容，解开注释，添加具体路径

# The java implementation to use.  Java 1.7+ required.
export JAVA_HOME=/usr/local/jdk

# Tell HBase whether it should manage it's own instance of Zookeeper or not.
export HBASE_MANAGES_ZK=false       #禁止内置zookeeper
 

修改hbase-site.xml
主要修改一下hbase在hdfs上的存储路径以及zookeeper的安装位置

<configuration>
    <!-- 指定hbase在HDFS上存储的路径 -->
    <property>
        <name>hbase.rootdir</name>
        <value>hdfs://mypc01:8020/hbase</value>
    </property>
    <!-- 指定hbase是分布式的 -->
    <property>
        <name>hbase.cluster.distributed</name>
        <value>true</value>
    </property>
    <!-- 指定zk的地址，多个用“,”分割 -->
    <property>
        <name>hbase.zookeeper.quorum</name>
        <value>mypc01:2181,mypc02:2181,mypc03:2181</value>
    </property>
     <!--将属性hbase.unsafe.stream.capability.enforce 改为true -->
    <property>
        <name>hbase.unsafe.stream.capability.enforce</name>
        <value>true</value>
    </property>
</configuration>

配置regionserver所在信息,如下表明有3台regionserver

[root@mypc01]#  vi $HBASE_HOME/conf/regionservers

删除localhost
添加:
	mypc01
    mypc02
    mypc03

配置备用hmaster,为了保证宕机时可以有备用

[root@mypc01 local]#  cd $HBASE_HOME/conf/
[root@mypc01 conf]#  echo "mypc02">> backup-masters

拷贝hadoop的配置文件到hbase

[root@mypc01 local]#  cd $HADOOP_HOME/etc/hadoop/
[root@mypc01 hadoop]#  cp core-site.xml hdfs-site.xml $HBASE_HOME/conf/

分发到其他机器

[root@mypc01 local]# scp -r /usr/local/hbase mypc02:/usr/local
[root@mypc01 local]# scp -r /usr/local/hbase mypc03:/usr/local
[root@mypc01 local]# scp /etc/profile mypc02:/etc/
[root@mypc01 local]# scp /etc/profile mypc03:/etc/

启动hbase

需首先启动hdfs和Zookeeper,因为hbase用到这个.
其中HRegionServer与HMaster为hbase相关进程

[root@mypc01 /]# start-hbase.sh
[root@mypc01 /]# jps
993 Application
3795 Jps
3479 HMaster
991 Application
3631 HRegionServer

hbase客户端

hbase shell

之后会进入hbase的命令行.

hbaseshell命令

Commands are grouped. Type 'help "COMMAND_GROUP"', (e.g. 'help "general"') for help on a command group.

COMMAND GROUPS:
  Group name: general
  Commands: status, table_help, version, whoami

  Group name: ddl
  Commands: alter, alter_async, alter_status, create, describe, disable, disable_all, drop, drop_all, enable, enable_all, exists, get_table, is_disa      bled, is_enabled, list, locate_region, show_filters

  Group name: namespace
  Commands: alter_namespace, create_namespace, describe_namespace, drop_namespace, list_namespace, list_namespace_tables

  Group name: dml
  Commands: append, count, delete, deleteall, get, get_counter, get_splits, incr, put, scan, truncate, truncate_preserve

  Group name: tools
  Commands: assign, balance_switch, balancer, balancer_enabled, catalogjanitor_enabled, catalogjanitor_run, catalogjanitor_switch, close_region, com      pact, compact_mob, compact_rs, flush, major_compact, major_compact_mob, merge_region, move, normalize, normalizer_enabled, normalizer_switch, split,       trace, unassign, wal_roll, zk_dump

  Group name: replication
  Commands: add_peer, append_peer_tableCFs, disable_peer, disable_table_replication, enable_peer, enable_table_replication, get_peer_config, list_pe      er_configs, list_peers, list_replicated_tables, remove_peer, remove_peer_tableCFs, set_peer_tableCFs, show_peer_tableCFs, update_peer_config

  Group name: snapshots
  Commands: clone_snapshot, delete_all_snapshot, delete_snapshot, list_snapshots, restore_snapshot, snapshot

  Group name: configuration
  Commands: update_all_config, update_config

  Group name: quotas
  Commands: list_quotas, set_quota

  Group name: security
  Commands: grant, list_security_capabilities, revoke, user_permission

  Group name: procedures
  Commands: abort_procedure, list_procedures

  Group name: visibility labels
  Commands: add_labels, clear_auths, get_auths, list_labels, set_auths, set_visibility

  Group name: rsgroup
  Commands: add_rsgroup, balance_rsgroup, get_rsgroup, get_server_rsgroup, get_table_rsgroup, list_rsgroups, move_servers_rsgroup, move_tables_rsgro      up, remove_rsgroup

table相关操作

- 直接创建
#ns3 是namespace,emp是表,base_info是列簇
hbase(main):037:0> create 'ns3:emp','base_info'
#shell界面会有结果提示,如下,表示操作的结果创建了一个叫做emp的表
=> Hbase::Table - ns3:emp

hbase(main):036:0> t = create 'ns3:employee','base_info'

#创建多个列族名
hbase(main):040:0> create 'ns3:emp1','f1','f2','f3'

#创建表时同时指定列族属性
hbase(main):043:0> create 'ns3:emp2',
  {NAME=>'f1',VERSIONS'10=>3,TTL=>2592000,BLOCKCACHE=>TRUE}

#创建表同时指定分片
  hbase(main):050:0> create 'ns3:emp=3', 'f1', SPLITS => ['10', '20', '30', '40']

有了表就要存数据

  hbase> put 'ns1:t1', 'r1', 'c1', 'value'
  hbase> put 't1', 'r1', 'c1', 'value'
  hbase> put 't1', 'r1', 'c1', 'value', ts1
  hbase> put 't1', 'r1', 'c1', 'value', {ATTRIBUTES=>{'mykey'=>'myvalue'}}
  hbase> put 't1', 'r1', 'c1', 'value', ts1, {ATTRIBUTES=>{'mykey'=>'myvalue'}}
  hbase> put 't1', 'r1', 'c1', 'value', ts1, {VISIBILITY=>'PRIVATE|SECRET'}

之后查看数据有两种方式 get和scan

  hbase> get 'ns1:t1', 'r1'
  hbase> get 't1', 'r1'
  hbase> get 't1', 'r1', {TIMERANGE => [ts1, ts2]}
  hbase> get 't1', 'r1', {COLUMN => 'c1'}
  hbase> get 't1', 'r1', {COLUMN => ['c1', 'c2', 'c3']}
  hbase> get 't1', 'r1', {COLUMN => 'c1', TIMESTAMP => ts1}
  hbase> get 't1', 'r1', {COLUMN => 'c1', TIMERANGE => [ts1, ts2], VERSIONS => 4}
  hbase> get 't1', 'r1', {COLUMN => 'c1', TIMESTAMP => ts1, VERSIONS => 4}
  hbase> get 't1', 'r1', {FILTER => "ValueFilter(=, 'binary:abc')"}
  hbase> get 't1', 'r1', 'c1'
  hbase> get 't1', 'r1', 'c1', 'c2'
  hbase> get 't1', 'r1', ['c1', 'c2']
  hbase> get 't1', 'r1', {COLUMN => 'c1', ATTRIBUTES => {'mykey'=>'myvalue'}}
  hbase> get 't1', 'r1', {COLUMN => 'c1', AUTHORIZATIONS => ['PRIVATE','SECRET']}
  hbase> get 't1', 'r1', {CONSISTENCY => 'TIMELINE'}
  hbase> get 't1', 'r1', {CONSISTENCY => 'TIMELINE', REGION_REPLICA_ID => 1}

Hbase Java API

新建maven项目导入依赖

<dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase-client</artifactId>
        <version>1.2.1</version>
</dependency>

连接测试

public class ConnectionDemo {
    public static void main(String[] args) throws IOException {
        // 1、获取配置对象
        Configuration conf = new Configuration();
        // 2、配置zookeeper的参数
        conf.set("hbase.zookeeper.quorum", "mypc01:2181,mypc02:2181,mypc03:2181");
        // 3、获取连接对象
        Connection connection = ConnectionFactory.createConnection(conf);
        // 4、获取一个DDL操作的hbase客户端Admin
        Admin admin = connection.getAdmin();
        //5 、创建一个namespace的描述器
        NamespaceDescriptor descriptor = NamespaceDescriptor.create("ns3").build();
        //6. 客户端进行提交创建
        admin.createNamespace(descriptor);
        //7. 关闭操作
        connection.close();
        admin.close();
    }
}