redis3.0集群搭建

最新推荐文章于 2024-03-22 21:45:14 发布

shenjianxz

最新推荐文章于 2024-03-22 21:45:14 发布

阅读量535

点赞数

分类专栏： redis

redis 专栏收录该内容

4 篇文章 0 订阅

订阅专栏

redis-cluster（集群）搭建过程

目的

本文章为redis-cluster搭建的step by step教程，具体原理，请见相关文档，在此不再详细讲解。

环境配置

本人在一台window 7上安装虚拟机，虚拟机中安装的是centos系统。

目标

在centos上搭建一个有6个节点的redis-cluster，端口号为 7000-7005

搭建步骤

新建文件夹

 
 mkdir /opt/app/redis_cluster
cd /opt/app/redis_cluster

下载、解压、编译

 
 $ wget http://download.redis.io/releases/redis-3.0.6.tar.gz
$ tar xzf redis-3.0.7.tar.gz
$ cd redis-3.0.7
$ make

创建六个文件夹

 
 [root@localhost redis-cluster]# mkdir 7000 7001 7002 7003 7004 7005

创建并配置redis.conf

分别对7000-7005文件夹中的6个文件修改对应的配置

 
 daemonize    yes                          //redis后台运行
pidfile  /var/run/redis_7000.pid          //pidfile文件对应7000,7001,7002
port  7000                                //端口7000,7002,7003
cluster-enabled  yes                      //开启集群  把注释#去掉
cluster-config-file  nodes_7000.conf      //集群的配置  配置文件首次启动自动生成 7000,7001,7002
bind 192.168.215.130                      //这里要绑定机器的IP
cluster-node-timeout  5000                //请求超时  设置5秒够了
appendonly  yes                           //aof日志开启  有需要就开启，它会每次写操作都记录一条日志</pre>

依次启动6个实例

启动7000实例

 
 [root@localhost redis_cluster]# redis-3.0.7/src/redis-server 7000/redis.conf &

成功启动之后，如图显示：

启动7001实例

 
 [root@localhost redis_cluster]# redis-3.0.7/src/redis-server 7001/redis.conf &

成功启动之后，如图显示：

依次类推，启动7002-7005实例。
启动完成如图：

将这些节点添加到集群中

连接到 7000 实例上，然后将 7001 - 7005 实例添加到集群中

其中红色框起来的就是为每一个节点负责的插槽（slot），redis-cluster总共有16384个插槽，目前我们创建了6个节点，所以均分得到这些区间段，将这些区间段分到每一个实例上去。
这个配置可以手动写到 7000节点的nodes.conf(nodes.conf文件只会在启动之后由redis创建)，写完之后再重启集群中的实例，然后再使用meet将这些节点添加到集群中。

 
 07673576597fd7d6cec9b5b0691571ce71e9098a 127.0.0.1:7003 master - 0 1472194995492 3 connected 8190-10919
1adc79c33fdfe908a41a97e044a4b072b0ba3042 127.0.0.1:7004 master - 0 1472194996298 4 connected 10920-13649
5b7a115c35ccbd266e4fe7071eb7375a53dd3dfd 127.0.0.1:7000 myself,master - 0 0 0 connected 0-2729
2a49fb3dead90778f9e1321654fae123496523d2 127.0.0.1:7002 master - 0 1472194996500 2 connected 5460-8189
10f4e506afa49558fcbd48ef081212ef11a6ad0c 127.0.0.1:7001 master - 0 1472194994987 1 connected 2730-5459
43f330e01fa94756556429bf105a3da21389620e 127.0.0.1:7005 master - 0 1472194996710 5 connected 13650-16383

必须更新7000-7005中所有的node.conf文件，没有更新的节点的状态会处于【fail】的。

测试集群

创建集群

前面已经准备好了搭建集群的Redis节点，接下来我们要把这些节点都串连起来搭建集群。官方提供了一个工具：redis-trib.rb(/usr/local/redis-3.2.1/src/redis-trib.rb) 看后缀就知道这鸟东西不能直接执行，它是用ruby写的一个程序，所以我们还得安装ruby.再用 gem 这个命令来安装 redis接口, gem是ruby的一个工具包.
为了方便，两台机器都安装.

 
 yum -y install ruby ruby-devel rubygems rpm-build
gem install redis

注意：这里需要修改gem源，并且淘宝的gem源已经不能用了，坑爹啊，现在要使用ruby-china。
https://gems.ruby-china.org/

添加ruby-china源：

 
 [root@zk1 software]# gem sources --add https://gems.ruby-china.org/ --remove https://rubygems.org/
[root@zk1 software]# gem sources -l
*** CURRENT SOURCES ***
https://gems.ruby-china.org/

redis-trib.rb

进入/usr/software/redis-3.2.5/src运行一下redis-trib.rb

 
 [root@zk2 src]# ./redis-trib.rb
Usage: redis-trib <command> <options> <arguments ...>
  create          host1:port1 ... hostN:portN
                  --replicas <arg>
  check           host:port
  info            host:port
  fix             host:port
                  --timeout <arg>
  reshard         host:port
                  --from <arg>
                  --to <arg>
                  --slots <arg>
                  --yes
                  --timeout <arg>
                  --pipeline <arg>
  rebalance       host:port
                  --weight <arg>
                  --auto-weights
                  --use-empty-masters
                  --timeout <arg>
                  --simulate
                  --pipeline <arg>
                  --threshold <arg>
  add-node        new_host:new_port existing_host:existing_port
                  --slave
                  --master-id <arg>
  del-node        host:port node_id
  set-timeout     host:port milliseconds
  call            host:port command arg arg .. arg
  import          host:port
                  --from <arg>
                  --copy
                  --replace
  help            (show this help)
For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.

看到这，应该明白了吧，就是靠上面这些操作完成redis集群搭建的.

确认所有的节点都启动，接下来使用参数create 创建 (在192.168.215.129中来创建)

 
 [root@zk1 src]# ./redis-trib.rb  create  --replicas  1  192.168.215.129:7000 192.168.215.129:7001  192.168.215.129:7002 192.168.215.130:7003  192.168.215.130:7004  192.168.215.130:7005
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.215.129:7000
192.168.215.130:7003
192.168.215.129:7001
Adding replica 192.168.215.130:7004 to 192.168.215.129:7000
Adding replica 192.168.215.129:7002 to 192.168.215.130:7003
Adding replica 192.168.215.130:7005 to 192.168.215.129:7001
M: 16518afbfcbd961aeb76ef1592007a3e7fe24b1b 192.168.215.129:7000
   slots:0-5460 (5461 slots) master
M: 524219969118a57ceaac753ecef7585f634cdf26 192.168.215.129:7001
   slots:10923-16383 (5461 slots) master
S: ea4519ff0083a13cef8262490ee9e61e5a4b14b1 192.168.215.129:7002
   replicates 82c0e591b9bc7a289026dff2873a254d1c49d285
M: 82c0e591b9bc7a289026dff2873a254d1c49d285 192.168.215.130:7003
   slots:5461-10922 (5462 slots) master
S: baf74dd89c0605d2a71a8d1d3706005ff668563b 192.168.215.130:7004
   replicates 16518afbfcbd961aeb76ef1592007a3e7fe24b1b
S: f8192314d2232e12ba9f558e9ecbfcc890f4fb73 192.168.215.130:7005
   replicates 524219969118a57ceaac753ecef7585f634cdf26
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join.....
>>> Performing Cluster Check (using node 192.168.215.129:7000)
M: 16518afbfcbd961aeb76ef1592007a3e7fe24b1b 192.168.215.129:7000
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: ea4519ff0083a13cef8262490ee9e61e5a4b14b1 192.168.215.129:7002
   slots: (0 slots) slave
   replicates 82c0e591b9bc7a289026dff2873a254d1c49d285
S: f8192314d2232e12ba9f558e9ecbfcc890f4fb73 192.168.215.130:7005
   slots: (0 slots) slave
   replicates 524219969118a57ceaac753ecef7585f634cdf26
S: baf74dd89c0605d2a71a8d1d3706005ff668563b 192.168.215.130:7004
   slots: (0 slots) slave
   replicates 16518afbfcbd961aeb76ef1592007a3e7fe24b1b
M: 524219969118a57ceaac753ecef7585f634cdf26 192.168.215.129:7001
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
M: 82c0e591b9bc7a289026dff2873a254d1c49d285 192.168.215.130:7003
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

解释下， –replicas 1 表示自动为每一个master节点分配一个slave节点上面有6个节点，程序会按照一定规则生成 3个master（主）3个slave(从)
前面已经提醒过的防火墙一定要开放监听的端口，否则会创建失败。

检查集群状态:

 
 [root@zk1 src]# ./redis-trib.rb check 192.168.215.129:7002
>>> Performing Cluster Check (using node 192.168.215.129:7002)
S: ea4519ff0083a13cef8262490ee9e61e5a4b14b1 192.168.215.129:7002
   slots: (0 slots) slave
   replicates 82c0e591b9bc7a289026dff2873a254d1c49d285
M: 82c0e591b9bc7a289026dff2873a254d1c49d285 192.168.215.130:7003
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: baf74dd89c0605d2a71a8d1d3706005ff668563b 192.168.215.130:7004
   slots: (0 slots) slave
   replicates 16518afbfcbd961aeb76ef1592007a3e7fe24b1b
M: 524219969118a57ceaac753ecef7585f634cdf26 192.168.215.129:7001
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
M: 16518afbfcbd961aeb76ef1592007a3e7fe24b1b 192.168.215.129:7000
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: f8192314d2232e12ba9f558e9ecbfcc890f4fb73 192.168.215.130:7005
   slots: (0 slots) slave
   replicates 524219969118a57ceaac753ecef7585f634cdf26
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

redis集群中数据分片是通过hash slot的方式实现的

测试集群

链接服务器

 
 [root@zk1 src]# ./redis-cli -c -p 7000 -h 192.168.215.129
192.168.215.129:7000>
192.168.215.129:7000> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_sent:1502
cluster_stats_messages_received:1502

set值

 
 192.168.215.129:7000> set name lbl
-> Redirected to slot [5798] located at 192.168.215.130:7003
OK
192.168.215.130:7003> get name
"lbl"

可见，重定向到了130节点7003端口。
原因是redis采用hash槽的方式分发key到不同节点，算法是crc(16)%16384。详细描述后续会单独写文章描述。
而且你会发现，当一次重定向以后，这个客户端就连接到了130:7003这个节点。

测试一个master宕机

将上面设置的name所在的130:7003kill掉，只剩了两个master和3个slave。你会发现cluster_current_epoch相比之前加了1，这是因为redis的主从关系，重新选了一次主。
然后get name发现，重定向了129:7002这个节点。

 
 [root@zk1 src]# ./redis-trib.rb check 192.168.215.129:7002
>>> Performing Cluster Check (using node 192.168.215.129:7002)
M: ea4519ff0083a13cef8262490ee9e61e5a4b14b1 192.168.215.129:7002
   slots:5461-10922 (5462 slots) master
   0 additional replica(s)
S: baf74dd89c0605d2a71a8d1d3706005ff668563b 192.168.215.130:7004
   slots: (0 slots) slave
   replicates 16518afbfcbd961aeb76ef1592007a3e7fe24b1b
M: 524219969118a57ceaac753ecef7585f634cdf26 192.168.215.129:7001
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
M: 16518afbfcbd961aeb76ef1592007a3e7fe24b1b 192.168.215.129:7000
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: f8192314d2232e12ba9f558e9ecbfcc890f4fb73 192.168.215.130:7005
   slots: (0 slots) slave
   replicates 524219969118a57ceaac753ecef7585f634cdf26
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@zk1 src]# ./redis-cli -c -p 7000 -h 192.168.215.129
192.168.215.129:7000> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:7
cluster_my_epoch:1
cluster_stats_messages_sent:2883
cluster_stats_messages_received:2675
192.168.215.129:7000> get name
-> Redirected to slot [5798] located at 192.168.215.129:7002
"lbl"

之所以会重定向到129:7002这个节点，是因为在kill之前129:7002是130:7003的slave：
下面这是在kill之前拷贝的./redis-trib.rb check的数据，注意replicates后的值

 
 1.  `S: ea4519ff0083a13cef8262490ee9e61e5a4b14b1 192.168.215.129:7002`
2.  `replicates 82c0e591b9bc7a289026dff2873a254d1c49d285`
3.  `M:  82c0e591b9bc7a289026dff2873a254d1c49d285  192.168.215.130:7003`
4.  `slots:5461-10922  (5462 slots) master`