NOSQL:Redis集群

集群概念

LB(负载均衡)和HA(高可用)

• redis集群能够实现的功能:数据分布式存储的高可用集群

集群环境

• redis服务器6台
• 管理主机1台
• 客户端1台

主机名	服务器IP	端口规划
client	192.168.4.50	无
redisA	192.168.4.51	6351
redisB	192.168.4.52	6352
redisC	192.168.4.53	6353
redisD	192.168.4.54	6354
redisE	192.168.4.55	6355
redisF	192.168.4.56	6356
MGM	192.168.4.57	无

创建集群

redis集群搭建

• 配置6台redis服务器
  –启动集群功能
  –查看服务信息
  redis安装请查看MYSQL:NoSQL(Redis服务)
  主机redisA上配置如下(按下列配置其余5台redis服务器):

vim /etc/redis/6379.conf
bind 192.168.4.51   #修改ip
port 6351           #修改端口（可选配置）
cluster-enabled yes #启用集群功能
cluster-config-file nodes-6379.conf #存储集群信息的配置文件
cluster-node-timeout 5000        #集群节点通信超时5s时间

rm  -rf  /var/lib/redis/6379/*   #清空数据
vim  +43 /etc/init.d/redis_6379
$CLIEXEC -h 192.168.4.51 -p 6351 shutdown

/etc/init.d/redis_6379 restart
netstat -utnlp  | grep redis-server
tcp  0  0 192.168.4.51:6351  0.0.0.0:*   LISTEN     21201/redis-server  
tcp  0  0 192.168.4.51:16351   0.0.0.0:*   LISTEN   21201/redis-server #16351集群通信端口,默认服务端口+10000 

ls /var/lib/redis/6379 #看集群文件
redis-cli  -h 192.168.4.51 -p 6351
192.168.4.51:6351> cluster info #查看集群信息
192.168.4.52:6352> cluster nodes #查看集群节点信息

部署管理主机

(也可以部署在任意一台redis服务器上)

• 部署ruby脚本运行环境
• 创建管理集群脚本redis-trib.rb(ruby写的脚本,所以需安装ruby)

yum -y install rubygems ruby #gem是ruby的一个工具包 
gem install redis-3.2.1.gem  #安装redis接口
mkdir /root/bin # 创建命令检索目录
tar -xf redis-4.0.8.tar.gz
cp redis-4.0.8/src/redis-trib.rb /root/bin
chmod  +x   /root/bin/redis-trib.rb
redis-trib.rb   help  #查看命令帮助

redis-trib脚本

redis-trib.rb <command> <options> <arguments...>

命令	描述
create	创建集群
check	检查集群
info	查看集群信息
reshard	重新分片
del-node	删除主机
add-node --slave	添加slave主机
add-node	添加master主机
rebalance	平均分配hash slots

在管理主机mgm57,创建集群

redis-trib.rb create  --replicas 1 \
> 192.168.4.51:6351  192.168.4.52:6352  192.168.4.53:6353 \
> 192.168.4.54:6354  192.168.4.55:6355  192.168.4.56:6356
redis-trib.rb info ip地址:端口 #查看集群信息
redis-trib.rb check ip地址:端口 # 检测及显示集群详细信息

访问集群

• 在客户端连接集群中的任意一台服务器存取数据

redis-cli  -c  -h 192.168.4.51 -p 6351 #-c代表集群模式
192.168.4.51:6351> set x 100  //存储
-> Redirected to slot [16287] located at 192.168.4.53:6353  //提示存储在53主机
OK
192.168.4.53:6353> keys *
1) "x"
192.168.4.53:6353>
192.168.4.53:6353> set y 200
OK
192.168.4.53:6353> keys *
1) "y"
2) "x"
192.168.4.53:6353> set z 300 //存储
-> Redirected to slot [8157] located at 192.168.4.52:6352 //提示存储在52主机
OK
192.168.4.52:6352> keys *  //在52主机查看数据 只有变量z 
1) "z"
192.168.4.52:6352> get x 
-> Redirected to slot [16287] located at 192.168.4.53:6353 //连接53主机获取数据
"100"
192.168.4.53:6353> keys *
1) "y"
2) "x"
192.168.4.53:6353> get z
-> Redirected to slot [8157] located at 192.168.4.52:6352
"300"
192.168.4.52:6352> set i 400
-> Redirected to slot [15759] located at 192.168.4.53:6353
OK
192.168.4.53:6353> set j 500
-> Redirected to slot [3564] located at 192.168.4.51:6351
OK

测试集群功能

故障切换测试

• 停止master主机的redis服务
  –master宕机后对应的slave自动被选举为master
  –原master后会自动配置为当前master的slave
  –宕机后节点还是存在只是fail状态
• 检测集群
  –在管理主机查看信息
  redis-trib.rb check 192.168.4.52:6352
redis-trib.rb info 192.168.4.52:6352

添加服务器

添加master服务器

• 部署一台新redis服务器
  –运行服务并启用集群配置
• 添加master主机步骤
  a.添加不指定主机角色,默认新主机被选为master
  b.添加的master主机,需要手动分配solts(槽)

1. 添加master主机(192.168.4.58)

redis-trib.rb  add-node  192.168.4.58:6358  192.168.4.53:6353  #执行添加命令
......
[OK] New node added correctly. #提示添加完成

redis-trib.rb info 192.168.4.53:6353  #查看集群信息
192.168.4.53:6353 (9e44139c...) -> 3 keys | 5461 slots | 1 slaves.
192.168.4.52:6352 (324e05df...) -> 2 keys | 5462 slots | 1 slaves.
192.168.4.58:6358 (4fe1fa46...) -> 0 keys | 0 slots | 0 slaves. #主服务器58
192.168.4.55:6355 (2d343a9d...) -> 3 keys | 5461 slots | 1 slaves.
[OK] 8 keys in 4 masters.
0.00 keys per slot on average.

2. 分配hash槽(slots)
   –移出hash槽个数
   –接受hash槽主机ID
   –移出hash槽主机ID
   –同意配置

redis-trib.rb  reshard   192.168.4.53:6353
How many slots do you want to move (from 1 to 16384)?4096   #拿出4096个hash 槽给主机192.168.4.58
What is the receiving node ID?  c5e0da48f335c46a2ec199faa99b830f537dd8a0   #主机192.168.4.58的id值
Source node #1:all      #从当前所有master服务器获取hash槽
Do you want to proceed with the proposed reshard plan (yes/no)?yes #同意以上配置 

添加slave服务器

• 部署一台新redis服务器
  –运行服务并启用集群配置
• 添加slave主机
  redis-trib.rb add-node --slave [--master id值] slave的ip地址:端口 192.168.4.52:6352
  (如果不指定主节点ID的话,会把新节点随机添加为从节点最少的主库)

redis-trib.rb add-node  --slave 192.168.4.59:6359  192.168.4.51:6351  #执行添加命令
......
[OK] New node added correctly.

redis-trib.rb  info  192.168.4.51:6351 #查看信息
192.168.4.55:6355 (2d343a9d...) -> 3 keys | 4096 slots | 1 slaves.
192.168.4.53:6353 (9e44139c...) -> 3 keys | 4096 slots | 1 slaves.
192.168.4.52:6352 (324e05df...) -> 2 keys | 4096 slots | 1 slaves.
192.168.4.58:6358 (4fe1fa46...) -> 5 keys | 4096 slots | 1 slaves. #有1个从服务器
[OK] 13 keys in 4 masters.
0.00 keys per slot on average.

移除服务器

移除slave服务器

• 从服务器没有hash槽,直接移除即可
• 移除是指定从服务器id值
  redis-trib.rb del-node Ip地址:端口 ID值 #执行移除命令

redis-trib.rb  del-node 192.168.4.51:6351 f6649ea99b2f01faca26217691222c17a3854381
>>> Removing node f6649ea99b2f01faca26217691222c17a3854381 from cluster 192.168.4.57:6351
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node. #停止移除服务的Redis服务

redis-trib.rb  info  192.168.4.51:6351
192.168.4.55:6355 (2d343a9d...) -> 3 keys | 4096 slots | 1 slaves.
192.168.4.53:6353 (9e44139c...) -> 3 keys | 4096 slots | 1 slaves.
192.168.4.52:6352 (324e05df...) -> 2 keys | 4096 slots | 1 slaves.
192.168.4.58:6358 (4fe1fa46...) -> 5 keys | 4096 slots | 0 slaves.  #58主机，没有从服务器
[OK] 13 keys in 4 masters.
0.00 keys per slot on average.

移除master服务器

(有slave的master服务器,释放hash后,slave会随机加入其他的master服务器)

• 释放占用的hash槽
  这里是移除主机58master

redis-trib.rb  reshard 192.168.4.53:6353 #指定是哪个集群
How many slots do you want to move (from 1 to 16384)?4096 //移除4096个数槽
What is the receiving node ID?  bc5c4e082a5a3391b634cf433a6486c867cfc44b  #要移动给谁的id即目标主机（这里可以随机写一个master的ID）  
Source node #1: c5e0da48f335c46a2ec199faa99b830f537dd8a0  #从谁那移动即源主机（这里写4.58的ID）  
Source node #2:done           #设置完毕
...
    Moving slot 12282 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
    Moving slot 12283 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
    Moving slot 12284 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
    Moving slot 12285 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
    Moving slot 12286 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
    Moving slot 12287 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
Do you want to proceed with the proposed reshard plan (yes/no)?yes #提交

redis-trib.rb  info  192.168.4.51:6351 #查看集群信息
192.168.4.55:6355 (2d343a9d...) -> 3 keys | 4096 slots | 1 slaves.
192.168.4.53:6353 (9e44139c...) -> 3 keys | 4096 slots | 1 slaves.
192.168.4.52:6352 (324e05df...) -> 2 keys | 4096 slots | 1 slaves.
192.168.4.58:6358 (4fe1fa46...) -> 0 keys | 0 slots    | 0 slaves. #零个槽
[OK] 13 keys in 4 masters.
0.00 keys per slot on average.

• 移除主机

redis-trib.rb del-node 192.168.4.53:6353 \ 
c5e0da48f335c46a2ec199faa99b830f537dd8a0    #删除谁+删除的id
>>> Removing node e081313ec843655d9bc5a17f3bed3de1dccb1d2b from cluster 192.168.4.51:6351
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.   #停止移除服务的Redis服务

redis-trib.rb  info  192.168.4.51:6351  #查看集群信息
192.168.4.55:6355 (2d343a9d...) -> 3 keys | 4096 slots | 1 slaves.
192.168.4.53:6353 (9e44139c...) -> 3 keys | 4096 slots | 1 slaves.
192.168.4.52:6352 (324e05df...) -> 2 keys | 4096 slots | 1 slaves.
[OK] 13 keys in 3 masters. #主服务器个数3台，没有58 
0.00 keys per slot on average.  

让移除的主机再次加入集群:cluster reset或者删除/var/lib/redis/6379/*
使集群中的master的槽(slots)平均分配:redis-trib.rb rebalance ip地址:端口(注:只会分配有slots的master,对没有slots的不分配)