大数据入门--HBase（一）环境搭建&常见命令

概念

架构角色

角色	作用
Master	是 HBase 集群中的主服务器，负责监控集群中的所有 RegionServer，并且是所有元数据更改的接口。通过zookeeper实现高可用。 Table : 创建（create）、删除（delete）、更新（alter） RegionServer : 分配regions到每个RegionServer，监控每个RegionServer的状态
Region Server	RegionServer 主要负责响应用户的请求，向 HDFS 读写数据。 Data : 获取（get）、添加（put）、更新（put）、删除（delete） Region : 切分Region（splitRegion），合并Region（compactRegion）
zookeeper	HBase 通过 Zookeeper 来做 Master 的高可用、RegionServer 的监控、元数据的入口以及集群配置的维护等工作。
HDFS	为 HBase 提供最终的底层数据存储服务，同时为 HBase 提供高可用的支持。

集群环境搭建

准备工作

• Hbase依赖zookeeper，所以需要先搭建zookeeper，参考此文
• Hbase对时间同步要求较高，建议使用ntp做时间同步，参考此文：ntp服务安装

安装

当前安装版本为：hbase-1.3.1

1. 解压安装包

tar -zxvf hbase-1.3.1-bin.tar.gz -C /opt/module/
cd /opt/module/hbase-1.3.1

2.修改配置文件

hbase-env.sh
添加

export JAVA_HOME=/opt/module/jdk1.8.0_144
# 不使用自带zk
export HBASE_MANAGES_ZK=false

hbase-site.xml

<!-- 我这里使用的是hadoop3,需要注意端口-->
<property>
    <name>hbase.rootdir</name>
    <value>hdfs://hadoop101:8020/HBase</value>
</property>
<property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
</property>
 <!-- 0.98 后的新变动，之前版本没有.port,默认端口为 60000 -->
<property>
    <name>hbase.master.port</name>
    <value>60000</value>
</property>
<property> 
    <name>hbase.zookeeper.quorum</name>
    <value>hadoop101:2181,hadoop102:2181,hadoop103:2181</value>
</property>
<!-- 使用外部zk时，可以不指定此配置项-->
<property> 
    <name>hbase.zookeeper.property.dataDir</name>
    <value>/opt/module/zookeeper-3.4.5/zkData</value>
</property>

regionserver

hadoop101
hadoop102
hadoop103

软连接 hadoop 配置文件到 HBase

ln -s /opt/module/hadoop-3.1.3/etc/hadoop/core-site.xml conf/core-site.xml
ln -s /opt/module/hadoop-3.1.3/etc/hadoop/hdfs-site.xml conf/hdfs-site.x

分布Hbase安装文件

xsync /opt/module/hbase-1.3.1

停起服务

单节点停起master

bin/hbase-daemon.sh start master
bin/hbase-daemon.sh stop master

检查是否启动成功,可以使用jps,出现如下进程证明master/regionserver是启动成功的

jps
---
3588 HMaster
3855 HRegionServer

单节点停起regionserver

bin/hbase-daemon.sh start regionserver
bin/hbase-daemon.sh stop regionserver

查看Hbase管理页面，网页地址：http://hadoop101:16010/

集群停起服务

# 执行start-hbase.sh的主机为master节点，同时根据conf/regionservers配置启动regionserver
bin/start-hbase.sh 
bin/stop-hbase.sh 

在浏览一下管理页面，三个节点都展示就OK了

常见命令

温馨提示
hbase命令行删除时backspace向后删除，delete向前删除，与平时习惯相反。
帮助

help
help ‘COMMAND’

hbase(main):001:0> help
HBase Shell, version 1.3.1, r930b9a55528fe45d8edce7af42fef2d35e77677a, Thu Apr  6 19:36:54 PDT 2017
Type 'help "COMMAND"', (e.g. 'help "get"' -- the quotes are necessary) for help on a specific command.
Commands are grouped. Type 'help "COMMAND_GROUP"', (e.g. 'help "general"') for help on a command group.

COMMAND GROUPS:
  Group name: general
  Commands: status, table_help, version, whoami

  Group name: ddl
  Commands: alter, alter_async, alter_status, create, describe, disable, disable_all, drop, drop_all, enable, enable_all, exists, get_table, is_disabled, is_enabled, list, locate_region, show_filters

  Group name: namespace
  Commands: alter_namespace, create_namespace, describe_namespace, drop_namespace, list_namespace, list_namespace_tables

  Group name: dml
  Commands: append, count, delete, deleteall, get, get_counter, get_splits, incr, put, scan, truncate, truncate_preserve

  Group name: tools
  Commands: assign, balance_switch, balancer, balancer_enabled, catalogjanitor_enabled, catalogjanitor_run, catalogjanitor_switch, close_region, compact, compact_rs, flush, major_compact, merge_region, move, normalize, normalizer_enabled, normalizer_switch, split, splitormerge_enabled, splitormerge_switch, trace, unassign, wal_roll, zk_dump

  Group name: replication
  Commands: add_peer, append_peer_tableCFs, disable_peer, disable_table_replication, enable_peer, enable_table_replication, get_peer_config, list_peer_configs, list_peers, list_replicated_tables, remove_peer, remove_peer_tableCFs, set_peer_tableCFs, show_peer_tableCFs

  Group name: snapshots
  Commands: clone_snapshot, delete_all_snapshot, delete_snapshot, delete_table_snapshots, list_snapshots, list_table_snapshots, restore_snapshot, snapshot

  Group name: configuration
  Commands: update_all_config, update_config

  Group name: quotas
  Commands: list_quotas, set_quota

  Group name: security
  Commands: grant, list_security_capabilities, revoke, user_permission

  Group name: procedures
  Commands: abort_procedure, list_procedures

  Group name: visibility labels
  Commands: add_labels, clear_auths, get_auths, list_labels, set_auths, set_visibility

SHELL USAGE:
Quote all names in HBase Shell such as table and column names.  Commas delimit
command parameters.  Type <RETURN> after entering a command to run it.
Dictionaries of configuration used in the creation and alteration of tables are
Ruby Hashes. They look like this:

  {'key1' => 'value1', 'key2' => 'value2', ...}

and are opened and closed with curley-braces.  Key/values are delimited by the
'=>' character combination.  Usually keys are predefined constants such as
NAME, VERSIONS, COMPRESSION, etc.  Constants do not need to be quoted.  Type
'Object.constants' to see a (messy) list of all constants in the environment.

If you are using binary keys or values and need to enter them in the shell, use
double-quote'd hexadecimal representation. For example:

  hbase> get 't1', "key\x03\x3f\xcd"
  hbase> get 't1', "key\003\023\011"
  hbase> put 't1', "test\xef\xff", 'f1:', "\x01\x33\x40"

The HBase shell is the (J)Ruby IRB with the above HBase-specific commands added.
For more on the HBase Shell, see http://hbase.apache.org/book.html

相关概念

• namespace：名称空间
• table：表
• Column Family ：列族
• Column：列

连接hbase

bin/hbase shell

DDL

Namespace

查看NS

hbase(main):001:0> list_namespace
NAMESPACE                                                                                                                                                                                                   
default                                                                                                                                                                                                     
hbase                                                                                                                                                                                                       
2 row(s) in 0.1660 seconds

创建NS

create_namespace 'study01'

删除NS
名称空间内不能有表，才允许删除NS

drop_namespace 'study01'

Table

建表

create 'study:test1', 'info1','info2'

查看表

list_namespace_tables 'study'

删除表：先禁用后删除

disable 'study01:tab01'
drop 'study01:tab01'

DML

插入/更新数据

hbase(main):012:0> put 'study:test1','0001','info1:name','zhangsan'