是什么
Hbase是一个非关系型的数据库
体系架构
client: 客户端
zookeeper
- 监控hmaster的状态,保证有且仅有一个活跃的hmaster,达到高可用.
- 存储meta表的寻址入口
- 感知regionserver的上下线信息,通知hmaster
Hmaster
- 为regionserver分配region
- 复杂regionserver的负载聚合
regionserver
- 管理本机的region
- 切分运行时变大的region
Hlog
记录hbase的操作,优先写入Hlog
region
一个表的一部分.
安装
以分布式安装为例
[root@mypc01 ~]# tar -zxvf hbase-1.2.1-bin.tar.gz -C /usr/local/
[root@mypc01 ~]# cd /usr/local/
[root@mypc01 local]# mv hbase-1.2.1 hbase
[root@mypc01 local]# vi /etc/profile
.......省略.......
#hbase environment
export HBASE_HOME=/usr/local/hbase
export PATH=$HBASE_HOME/bin:$PATH
[root@mypc01 local]# source /etc/profile
[root@mypc01 local]# hbase version
禁用hbase自带zookeeper,用自己搭建的,需要修改hbase-env.sh文件
[root@mypc01 local]# vi $HBASE_HOME/conf/hbase-env.sh
#找到下面内容,解开注释,添加具体路径
# The java implementation to use. Java 1.7+ required.
export JAVA_HOME=/usr/local/jdk
# Tell HBase whether it should manage it's own instance of Zookeeper or not.
export HBASE_MANAGES_ZK=false #禁止内置zookeeper
修改hbase-site.xml
主要修改一下hbase在hdfs上的存储路径以及zookeeper的安装位置
<configuration>
<!-- 指定hbase在HDFS上存储的路径 -->
<property>
<name>hbase.rootdir</name>
<value>hdfs://mypc01:8020/hbase</value>
</property>
<!-- 指定hbase是分布式的 -->
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<!-- 指定zk的地址,多个用“,”分割 -->
<property>
<name>hbase.zookeeper.quorum</name>
<value>mypc01:2181,mypc02:2181,mypc03:2181</value>
</property>
<!--将属性hbase.unsafe.stream.capability.enforce 改为true -->
<property>
<name>hbase.unsafe.stream.capability.enforce</name>
<value>true</value>
</property>
</configuration>
配置regionserver所在信息,如下表明有3台regionserver
[root@mypc01]# vi $HBASE_HOME/conf/regionservers
删除localhost
添加:
mypc01
mypc02
mypc03
配置备用hmaster,为了保证宕机时可以有备用
[root@mypc01 local]# cd $HBASE_HOME/conf/
[root@mypc01 conf]# echo "mypc02">> backup-masters
拷贝hadoop的配置文件到hbase
[root@mypc01 local]# cd $HADOOP_HOME/etc/hadoop/
[root@mypc01 hadoop]# cp core-site.xml hdfs-site.xml $HBASE_HOME/conf/
分发到其他机器
[root@mypc01 local]# scp -r /usr/local/hbase mypc02:/usr/local
[root@mypc01 local]# scp -r /usr/local/hbase mypc03:/usr/local
[root@mypc01 local]# scp /etc/profile mypc02:/etc/
[root@mypc01 local]# scp /etc/profile mypc03:/etc/
启动hbase
需首先启动hdfs和Zookeeper,因为hbase用到这个.
其中HRegionServer与HMaster为hbase相关进程
[root@mypc01 /]# start-hbase.sh
[root@mypc01 /]# jps
993 Application
3795 Jps
3479 HMaster
991 Application
3631 HRegionServer
hbase客户端
hbase shell
之后会进入hbase的命令行.
hbaseshell命令
Commands are grouped. Type 'help "COMMAND_GROUP"', (e.g. 'help "general"') for help on a command group.
COMMAND GROUPS:
Group name: general
Commands: status, table_help, version, whoami
Group name: ddl
Commands: alter, alter_async, alter_status, create, describe, disable, disable_all, drop, drop_all, enable, enable_all, exists, get_table, is_disa bled, is_enabled, list, locate_region, show_filters
Group name: namespace
Commands: alter_namespace, create_namespace, describe_namespace, drop_namespace, list_namespace, list_namespace_tables
Group name: dml
Commands: append, count, delete, deleteall, get, get_counter, get_splits, incr, put, scan, truncate, truncate_preserve
Group name: tools
Commands: assign, balance_switch, balancer, balancer_enabled, catalogjanitor_enabled, catalogjanitor_run, catalogjanitor_switch, close_region, com pact, compact_mob, compact_rs, flush, major_compact, major_compact_mob, merge_region, move, normalize, normalizer_enabled, normalizer_switch, split, trace, unassign, wal_roll, zk_dump
Group name: replication
Commands: add_peer, append_peer_tableCFs, disable_peer, disable_table_replication, enable_peer, enable_table_replication, get_peer_config, list_pe er_configs, list_peers, list_replicated_tables, remove_peer, remove_peer_tableCFs, set_peer_tableCFs, show_peer_tableCFs, update_peer_config
Group name: snapshots
Commands: clone_snapshot, delete_all_snapshot, delete_snapshot, list_snapshots, restore_snapshot, snapshot
Group name: configuration
Commands: update_all_config, update_config
Group name: quotas
Commands: list_quotas, set_quota
Group name: security
Commands: grant, list_security_capabilities, revoke, user_permission
Group name: procedures
Commands: abort_procedure, list_procedures
Group name: visibility labels
Commands: add_labels, clear_auths, get_auths, list_labels, set_auths, set_visibility
Group name: rsgroup
Commands: add_rsgroup, balance_rsgroup, get_rsgroup, get_server_rsgroup, get_table_rsgroup, list_rsgroups, move_servers_rsgroup, move_tables_rsgro up, remove_rsgroup
table相关操作
- 直接创建
#ns3 是namespace,emp是表,base_info是列簇
hbase(main):037:0> create 'ns3:emp','base_info'
#shell界面会有结果提示,如下,表示操作的结果创建了一个叫做emp的表
=> Hbase::Table - ns3:emp
hbase(main):036:0> t = create 'ns3:employee','base_info'
#创建多个列族名
hbase(main):040:0> create 'ns3:emp1','f1','f2','f3'
#创建表时同时指定列族属性
hbase(main):043:0> create 'ns3:emp2',
{NAME=>'f1',VERSIONS'10=>3,TTL=>2592000,BLOCKCACHE=>TRUE}
#创建表同时指定分片
hbase(main):050:0> create 'ns3:emp=3', 'f1', SPLITS => ['10', '20', '30', '40']
有了表就要存数据
hbase> put 'ns1:t1', 'r1', 'c1', 'value'
hbase> put 't1', 'r1', 'c1', 'value'
hbase> put 't1', 'r1', 'c1', 'value', ts1
hbase> put 't1', 'r1', 'c1', 'value', {ATTRIBUTES=>{'mykey'=>'myvalue'}}
hbase> put 't1', 'r1', 'c1', 'value', ts1, {ATTRIBUTES=>{'mykey'=>'myvalue'}}
hbase> put 't1', 'r1', 'c1', 'value', ts1, {VISIBILITY=>'PRIVATE|SECRET'}
之后查看数据有两种方式 get和scan
hbase> get 'ns1:t1', 'r1'
hbase> get 't1', 'r1'
hbase> get 't1', 'r1', {TIMERANGE => [ts1, ts2]}
hbase> get 't1', 'r1', {COLUMN => 'c1'}
hbase> get 't1', 'r1', {COLUMN => ['c1', 'c2', 'c3']}
hbase> get 't1', 'r1', {COLUMN => 'c1', TIMESTAMP => ts1}
hbase> get 't1', 'r1', {COLUMN => 'c1', TIMERANGE => [ts1, ts2], VERSIONS => 4}
hbase> get 't1', 'r1', {COLUMN => 'c1', TIMESTAMP => ts1, VERSIONS => 4}
hbase> get 't1', 'r1', {FILTER => "ValueFilter(=, 'binary:abc')"}
hbase> get 't1', 'r1', 'c1'
hbase> get 't1', 'r1', 'c1', 'c2'
hbase> get 't1', 'r1', ['c1', 'c2']
hbase> get 't1', 'r1', {COLUMN => 'c1', ATTRIBUTES => {'mykey'=>'myvalue'}}
hbase> get 't1', 'r1', {COLUMN => 'c1', AUTHORIZATIONS => ['PRIVATE','SECRET']}
hbase> get 't1', 'r1', {CONSISTENCY => 'TIMELINE'}
hbase> get 't1', 'r1', {CONSISTENCY => 'TIMELINE', REGION_REPLICA_ID => 1}
Hbase Java API
新建maven项目导入依赖
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>1.2.1</version>
</dependency>
连接测试
public class ConnectionDemo {
public static void main(String[] args) throws IOException {
// 1、获取配置对象
Configuration conf = new Configuration();
// 2、配置zookeeper的参数
conf.set("hbase.zookeeper.quorum", "mypc01:2181,mypc02:2181,mypc03:2181");
// 3、获取连接对象
Connection connection = ConnectionFactory.createConnection(conf);
// 4、获取一个DDL操作的hbase客户端Admin
Admin admin = connection.getAdmin();
//5 、创建一个namespace的描述器
NamespaceDescriptor descriptor = NamespaceDescriptor.create("ns3").build();
//6. 客户端进行提交创建
admin.createNamespace(descriptor);
//7. 关闭操作
connection.close();
admin.close();
}
}