【搭建 Hbase 集群】


在内容开始之前需要了解一下几个方面的知识会更好的帮助学习和搭建Hbase集群

  • Hadoop:Hbase 是建立在 Hadoop基础之上的分布式数据库。了解 Hadoop 的基本概念和架构将有助于您理解 Hbase 的工作原理。

  • 分布式系统:Hbase是一个分布式数据库,它将数据存储在多台服务器上。了解分布式系统的概念、优点、挑战和常见的解决方案,将帮助您更好地理解和管理
    Hbase集群。

  • ZooKeeper:Hbase 使用 ZooKeeper 来协调和管理集群中的各个节点。了解 ZooKeeper的基本概念、角色和功能,以及如何安装、配置和运行 ZooKeeper,将有助于您正确地配置和管理 Hbase 的
    ZooKeeper。

  • 网络和主机配置:在搭建 Hbase集群之前,需要确保服务器之间能够相互通信,并具有足够的网络带宽和稳定性。此外,还需要了解如何设置主机名解析和网络配置,以便在集群中使用主机名进行通信。

  • 数据模型和表设计:Hbase 使用键值对存储和访问数据,了解如何设计良好的数据模型和表结构将有助于提高性能和查询效率。

  • 安全性配置:在生产环境中搭建 Hbase集群时,安全性是一个重要的考虑因素。了解如何配置安全认证和访问控制,以保护您的数据和集群是很重要的。

一、准备工作

  1. Hadoop 官方网站 https://archive.apache.org/dist/hadoop/common/
  2. Hbase:官方网站 https://archive.apache.org/dist/hbase/
  3. ZooKeeper:Hbase 使用 ZooKeeper进行协调和管理。官方网站:https://archive.apache.org/dist/zookeeper/
  4. HDFS:Hbase 使用 Hadoop HDFS存储其数据。因此,您需要安装和配置 Hadoop HDFS。一般情况下,Hadoop 的安装包中已经包含了 HDFS。
  5. JDK:确保服务器上安装了 Java Development Kit(JDK)。Hadoop 和 Hbase 都需要运行在 Java环境中,因此需要安装并正确配置 JDK。

我们在这里使用的是一下版本:

lvrensun-server1

[root@lvrensun-server1 soft]# pwd
/root/soft
[root@lvrensun-server1 soft]# ll -h
总用量 700M
-rw-r--r--.  1 root root 396M 22 23:09 hadoop-2.10.2.tar.gz
-rw-r--r--.  1 root root  79M 130 01:50 hbase-0.98.6-hadoop2-bin.tar.gz
-rw-r--r--.  1 root root 177M 316 2017 jdk-8u131-linux-x64.tar.gz
-rw-r--r--.  1 root root  16M 130 01:50 zookeeper-3.4.5.tar.gz
[root@lvrensun-server1 soft]# 

lvrensun-server2

[root@lvrensun-server2 soft]# pwd
/root/soft
[root@lvrensun-server2 soft]# ll -h
总用量 0
[root@lvrensun-server2 soft]# 

lvrensun-server3

[root@lvrensun-server3 soft]# pwd
/root/soft
[root@lvrensun-server3 soft]# ll -h
总用量 0
[root@lvrensun-server3 soft]# 

可以看到上方显示有三台服务器分别是:

hostnameIP资源路径是否有资源包
lvrensun-server1192.168.0.11/root/soft
lvrensun-server2192.168.0.22/root/soft
lvrensun-server3192.168.0.33/root/soft

二、三台服务器之间的 SSH 免密登录

1.修改hosts文件添加DNS映射

vim /etc/hosts

如下图所示
在这里插入图片描述

2.在每台服务器上生成 SSH 密钥对

[root@lvrensun-server1 soft]# ssh-keygen -t rsa
[root@lvrensun-server2 soft]# ssh-keygen -t rsa
[root@lvrensun-server3 soft]# ssh-keygen -t rsa

在生成密钥对时,您可以选择存储位置和设置密钥的密码。如果留空密码,则可以实现完全无密码的免密登录。

3.将公共密钥(通常为 ~/.ssh/id_rsa.pub)复制到目标服务器上

[root@lvrensun-server1 ~]# ssh-copy-id root@192.168.0.11
[root@lvrensun-server1 ~]# ssh-copy-id root@192.168.0.22
[root@lvrensun-server1 ~]# ssh-copy-id root@192.168.0.33
[root@lvrensun-server2 ~]# ssh-copy-id root@192.168.0.11
[root@lvrensun-server2 ~]# ssh-copy-id root@192.168.0.22
[root@lvrensun-server2 ~]# ssh-copy-id root@192.168.0.33
[root@lvrensun-server3 ~]# ssh-copy-id root@192.168.0.11
[root@lvrensun-server3 ~]# ssh-copy-id root@192.168.0.22
[root@lvrensun-server3 ~]# ssh-copy-id root@192.168.0.33

前两步其中一台服务器详细执行步骤如下:

[root@lvrensun-server1 soft]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:QeDtQ6BMG5mIM2rk2DjM87H5Qe9uCv2i16bF55CI3fI root@lvrensun-server1
The key's randomart image is:
+---[RSA 2048]----+
| . .ooo..        |
|+..oo= +         |
|B=  + . +        |
|=*.. . o .       |
|..o = . S        |
|   ++.+...       |
|   o.=+* .       |
|    .+==+        |
|   .ooBE .       |
+----[SHA256]-----+
[root@lvrensun-server1 ~]# cat ~/.ssh/id_rsa.pub 
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNW0i7f2CAa/rq55WqUtJBc8igbOSTrsJGlFjX2s1XmONL1nPHk3eKEy1HrqC5kpEwgjYcIawtIq/apERkjqeGNRbjV9/Sq7zUvwiONA1+WJTwI80RyH84ZhV6aHuXTrly+9cx8UF1TbVRh0WgWZcjzFZdzDVIcB18zsBmWXZsGZxxVYD4fjkfXjMKJwOcAGXlUWb3J7O30uB9GcJhasTN98QcFmVp+cGldNJbTlXgUSUffD0HTOAPhlWDgWI3dFq+jCnCI4XzugyF0vU4dv81FiBTgcTxJ25tJvnw2PwtxhSEpklfK9UuYLBrOlDMsDBlkKBITTddJkLOvCsLr1Sr root@lvrensun-server1
[root@lvrensun-server1 ~]# ssh-copy-id root@192.168.0.11
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host '192.168.0.11 (192.168.0.11)' can't be established.
ECDSA key fingerprint is SHA256:SPQuwBgvdNmYL9PVBgcGTkuynS9iqvn3pBvXv6ZcePw.
ECDSA key fingerprint is MD5:db:a1:53:83:6c:52:e5:6c:d8:75:6a:05:47:b7:26:5b.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.0.11's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'root@192.168.0.11'"
and check to make sure that only the key(s) you wanted were added.

[root@lvrensun-server1 ~]# ssh-copy-id root@192.168.0.22
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host '192.168.0.22 (192.168.0.22)' can't be established.
ECDSA key fingerprint is SHA256:SPQuwBgvdNmYL9PVBgcGTkuynS9iqvn3pBvXv6ZcePw.
ECDSA key fingerprint is MD5:db:a1:53:83:6c:52:e5:6c:d8:75:6a:05:47:b7:26:5b.
Are you sure you want to continue connecting (yes/no)? yes   
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.0.22's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'root@192.168.0.22'"
and check to make sure that only the key(s) you wanted were added.

[root@lvrensun-server1 ~]# ssh-copy-id root@192.168.0.33
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host '192.168.0.33 (192.168.0.33)' can't be established.
ECDSA key fingerprint is SHA256:SPQuwBgvdNmYL9PVBgcGTkuynS9iqvn3pBvXv6ZcePw.
ECDSA key fingerprint is MD5:db:a1:53:83:6c:52:e5:6c:d8:75:6a:05:47:b7:26:5b.
Are you sure you want to continue connecting (yes/no)? yes   
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.0.33's password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'root@192.168.0.33'"
and check to make sure that only the key(s) you wanted were added.

[root@lvrensun-server1 ~]# 

4.从本地使用 SSH 命令无需密码连接到目标服务器

lvrensun-server1

[root@lvrensun-server1 ~]# ssh lvrensun-server1
Last login: Fri Feb  2 12:54:38 2024 from lvrensun-server2
[root@lvrensun-server1 ~]# exit
登出
Connection to lvrensun-server1 closed.
[root@lvrensun-server1 ~]# ssh lvrensun-server2
Last login: Fri Feb  2 12:54:51 2024 from lvrensun-server2
[root@lvrensun-server2 ~]# exit
登出
Connection to lvrensun-server2 closed.
[root@lvrensun-server1 ~]# ssh lvrensun-server3
Last login: Fri Feb  2 12:55:14 2024 from lvrensun-server2
[root@lvrensun-server3 ~]# 

lvrensun-server2

[root@lvrensun-server2 ~]# ssh lvrensun-server1
Last login: Fri Feb  2 12:53:50 2024 from lvrensun-server2
[root@lvrensun-server1 ~]# exit
登出
Connection to lvrensun-server1 closed.
[root@lvrensun-server2 ~]# ssh lvrensun-server2
Last login: Fri Feb  2 12:54:09 2024 from lvrensun-server2
[root@lvrensun-server2 ~]# exit
登出
Connection to lvrensun-server2 closed.
[root@lvrensun-server2 ~]# ssh lvrensun-server3
Last login: Fri Feb  2 12:54:21 2024 from lvrensun-server2
[root@lvrensun-server3 ~]# 

lvrensun-server3

[root@lvrensun-server3 ~]# ssh lvrensun-server1
Last login: Fri Feb  2 12:56:09 2024 from lvrensun-server1
[root@lvrensun-server1 ~]# exit
登出
Connection to lvrensun-server1 closed.
[root@lvrensun-server3 ~]# ssh lvrensun-server2
Last login: Fri Feb  2 12:56:19 2024 from lvrensun-server1
[root@lvrensun-server2 ~]# exit
登出
Connection to lvrensun-server2 closed.
[root@lvrensun-server3 ~]# ssh lvrensun-server3
Last login: Fri Feb  2 12:56:27 2024 from lvrensun-server1
[root@lvrensun-server3 ~]# 

二、安装JDK

Hadoop 和 Hbase 都需要运行在 Java环境中因此JDK属于基础环境

1.执行解压更名

 tar -xzvf jdk-8u131-linux-x64.tar.gz
 mv jdk-1.8.0-u131/ jdk1.8

如下:

-rw-r--r--. 1 root root  34687765 1月  30 01:50 hadoop-2.10.2-site.tar.gz
-rw-r--r--. 1 root root  82347017 1月  30 01:50 hbase-0.98.6-hadoop2-bin.tar.gz
drwxr-xr-x. 8   10  143       255 315 2017 jdk1.8
-rw-r--r--. 1 root root 185540433 3月  16 2017 jdk-8u131-linux-x64.tar.gz
-rw-r--r--. 1 root root  16402010 1月  30 01:50 zookeeper-3.4.5.tar.gz

2.配置环境 source生效

vim /etc/profile

[root@lvrensun-server1 soft]# vim /etc/profile

添加配置

export JAVA_HOME=/root/soft/jdk1.8
export PATH=$PATH:$JAVA_HOME/bin

source生效

[root@lvrensun-server1 soft]# source /etc/profile
[root@lvrensun-server1 soft]# java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
[root@lvrensun-server1 soft]# 

3.分发环境配置

分发JDK

[root@lvrensun-server1 soft]# scp -r jdk1.8/ lvrensun-server2:/root/soft/
[root@lvrensun-server1 soft]# scp -r jdk1.8/ lvrensun-server3:/root/soft/

分发环境变量

[root@lvrensun-server1 soft]# scp /etc/profile lvrensun-server2:/etc/
[root@lvrensun-server1 soft]# scp /etc/profile lvrensun-server3:/etc/

再到 lvresun-server2、lvrensun-server3 source生效环境变量

[root@lvrensun-server2 ~]# source /etc/profile
[root@lvrensun-server2 ~]# java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
[root@lvrensun-server2 ~]# 
[root@lvrensun-server3 soft]# source /etc/profile
[root@lvrensun-server3 soft]# java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
[root@lvrensun-server3 soft]# 

三、安装ZooKeeper 来协调和管理集群中的各个节点

1.解压更名

[root@lvrensun-server1 soft]# tar -xzvf zookeeper-3.4.5.tar.gz
.......
-rw-r--r--.  1 root root   34687765 1月  30 01:50 hadoop-2.10.2-site.tar.gz
-rw-r--r--.  1 root root   82347017 1月  30 01:50 hbase-0.98.6-hadoop2-bin.tar.gz
drwxr-xr-x.  8   10   143       255 315 2017 jdk1.8
-rw-r--r--.  1 root root  185540433 3月  16 2017 jdk-8u131-linux-x64.tar.gz
drwxr-xr-x. 10  501 games      4096 115 2012 zookeeper-3.4.5
-rw-r--r--.  1 root root   16402010 1月  30 01:50 zookeeper-3.4.5.tar.gz
[root@lvrensun-server1 soft]# mv zookeeper-3.4.5 zookeeper
[root@lvrensun-server1 soft]# ll
总用量 311512
-rw-r--r--.  1 root root   34687765 1月  30 01:50 hadoop-2.10.2-site.tar.gz
-rw-r--r--.  1 root root   82347017 1月  30 01:50 hbase-0.98.6-hadoop2-bin.tar.gz
drwxr-xr-x.  8   10   143       255 315 2017 jdk1.8
-rw-r--r--.  1 root root  185540433 3月  16 2017 jdk-8u131-linux-x64.tar.gz
drwxr-xr-x. 10  501 games      4096 115 2012 zookeeper
-rw-r--r--.  1 root root   16402010 1月  30 01:50 zookeeper-3.4.5.tar.gz
[root@lvrensun-server1 soft]# 

2.修改配置文件

2-1.配置文件更名修改配置

[root@lvrensun-server1 conf]# pwd
/root/soft/zookeeper/conf
[root@lvrensun-server1 conf]# mv zoo_sample.cfg zoo.cfg
[root@lvrensun-server1 conf]# vim zoo.cfg

zoo.cfg 修改内容如下:


# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/root/soft/zookeeper/data
dataLogDir=/root/soft/zookeeper/log
# the port at which the clients will connect
clientPort=2181
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

server.1=lvrensun-server1:2888:2299
server.2=lvrensun-server2:2888:2299
server.3=lvrensun-server3:2888:2299

2-2.根据配置创建数据所需文件夹

[root@lvrensun-server1 conf]# mkdir -p /root/soft/zookeeper/data
[root@lvrensun-server1 conf]# mkdir -p /root/soft/zookeeper/log
[root@lvrensun-server1 conf]# ll ../
总用量 1488
drwxr-xr-x.  2  501 games     149 22 13:18 bin
-rw-r--r--.  1  501 games   75988 10月  1 2012 build.xml
-rw-r--r--.  1  501 games   70223 10月  1 2012 CHANGES.txt
drwxr-xr-x.  2  501 games      70 22 13:32 conf
drwxr-xr-x. 10  501 games     130 22 13:18 contrib
drwxr-xr-x.  2 root root        6 22 13:34 data
drwxr-xr-x.  2  501 games    4096 22 13:18 dist-maven
drwxr-xr-x.  6  501 games    4096 22 13:18 docs
-rw-r--r--.  1  501 games    1953 10月  1 2012 ivysettings.xml
-rw-r--r--.  1  501 games    3120 10月  1 2012 ivy.xml
drwxr-xr-x.  4  501 games     235 22 13:18 lib
-rw-r--r--.  1  501 games   11358 10月  1 2012 LICENSE.txt
drwxr-xr-x.  2 root root        6 22 13:34 log
-rw-r--r--.  1  501 games     170 10月  1 2012 NOTICE.txt
-rw-r--r--.  1  501 games    1770 10月  1 2012 README_packaging.txt
-rw-r--r--.  1  501 games    1585 10月  1 2012 README.txt
drwxr-xr-x.  5  501 games      47 22 13:18 recipes
drwxr-xr-x.  8  501 games     154 22 13:18 src
-rw-r--r--.  1  501 games 1315806 11月  5 2012 zookeeper-3.4.5.jar
-rw-r--r--.  1  501 games     833 11月  5 2012 zookeeper-3.4.5.jar.asc
-rw-r--r--.  1  501 games      33 11月  5 2012 zookeeper-3.4.5.jar.md5
-rw-r--r--.  1  501 games      41 11月  5 2012 zookeeper-3.4.5.jar.sha1
[root@lvrensun-server1 conf]# 

3.配置环境

[root@lvrensun-server1 zookeeper]# vim /etc/profile

添加内容:

export ZOOKEEPER_HOME=/root/soft/zookeeper
export PATH=$PATH:$ZOOKEEPER_HOME/bin

生效

[root@lvrensun-server1 zookeeper]# source /etc/profile

4.分发环境

分发包:

[root@lvrensun-server1 soft]# pwd
/root/soft
[root@lvrensun-server1 soft]# scp -r zookeeper lvrensun-server2:/root/soft/
......
[root@lvrensun-server1 soft]# scp -r zookeeper lvrensun-server3:/root/soft/
......
[root@lvrensun-server1 soft]# 

5.分发环境

5-1.文件远程拷贝

[root@lvrensun-server1 soft]# scp /etc/profile lvrensun-server2:/etc/
[root@lvrensun-server1 soft]# scp /etc/profile lvrensun-server3:/etc/

5-2. lvresun-server2、lvrensun-server3 source生效环境变量

[root@lvrensun-server2 soft]# source /etc/profile
[root@lvrensun-server3 soft]# source /etc/profile

6. 每台zookeeper服务分配myid

[root@lvrensun-server1 bin]# echo "1" > /root/soft/zookeeper/data/myid
[root@lvrensun-server2 bin]# echo "2" > /root/soft/zookeeper/data/myid
[root@lvrensun-server3 bin]# echo "3" > /root/soft/zookeeper/data/myid

其中 myid中的 1、2、3 分别对应的是 zoo.cfg 文件中配置项server.1、server.2、server.3中一一对应的数字,知道就行了,对应即可

7.启动验证

[root@lvrensun-server1 bin]# cd /root/soft/zookeeper/bin/
[root@lvrensun-server1 bin]# zkServer.sh start
JMX enabled by default
Using config: /root/soft/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@lvrensun-server1 bin]# zkServer.sh status
JMX enabled by default
Using config: /root/soft/zookeeper/bin/../conf/zoo.cfg
Mode: follower
[root@lvrensun-server1 bin]# 

[root@lvrensun-server2 bin]# cd /root/soft/zookeeper/bin/
[root@lvrensun-server2 bin]# zkServer.sh start
JMX enabled by default
Using config: /root/soft/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@lvrensun-server2 bin]# zkServer.sh status
JMX enabled by default
Using config: /root/soft/zookeeper/bin/../conf/zoo.cfg
Mode: follower
[root@lvrensun-server2 bin]# 

[root@lvrensun-server3 bin]# cd /root/soft/zookeeper//bin/
[root@lvrensun-server3 bin]# zkServer.sh start
JMX enabled by default
Using config: /root/soft/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@lvrensun-server3 bin]# zkServer.sh status
JMX enabled by default
Using config: /root/soft/zookeeper/bin/../conf/zoo.cfg
Mode: leader

启动一切正产 lvrensun-server3 为主节点

8.验证宕机选举是否正常

把主服务器停掉

[root@lvrensun-server3 bin]# ./zkServer.sh stop
JMX enabled by default
Using config: /root/soft/zookeeper/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
[root@lvrensun-server3 bin]# 

查看其他两台状态

[root@lvrensun-server1 bin]# zkServer.sh status
JMX enabled by default
Using config: /root/soft/zookeeper/bin/../conf/zoo.cfg
Mode: follower
[root@lvrensun-server1 bin]# 

[root@lvrensun-server2 bin]# zkServer.sh status
JMX enabled by default
Using config: /root/soft/zookeeper/bin/../conf/zoo.cfg
Mode: leader
[root@lvrensun-server2 bin]# 

可以看到 lvrensun-server2 服务器中zookeeper节点已经被选为主服务节点

四、安装 Hadoop Hbase 是建立在 Hadoop基础之上的分布式数据库

  • 23
    点赞
  • 21
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值