集群拓扑结构
拓扑结构:
Redis Cluster物理结构:
角色 | ip | port | 别名 |
---|---|---|---|
master-1 | 127.0.0.1 | 6480 | master-1或者6380节点 |
master-2 | 127.0.0.1 | 6481 | master-2或者6381节点 |
master-3 | 127.0.0.1 | 6482 | master-3或者6382节点 |
slave-1 | 127.0.0.1 | 6483 | slave-1或者6483 节点 |
slave-2 | 127.0.0.1 | 6484 | slave-2或者6484 节点 |
slave-3 | 127.0.0.1 | 6485 | slave-3或者6485 节点 |
准备节点
其实每个节点的配置文件都是差不多的,主要是端口不同,存放的数据,日志文件等名称不同而已。建议将集群的节点配置目录统一放置,一般划分为conf、data、log三个目录。
配置6480,6481,6482,6483,6484,6485节点配置文件:
//redis-6480.conf
//端口
port 6480
//开启集群模式
cluster-enabled yes
//集群内部配置文件
cluster-config-file nodes-6480.conf
//节点超时时间
cluster-node-timeout 15000
其他配置项和单机模式配置一样
启动所有节点:
redis-server redis-6480.conf
redis-server redis-6481.conf
redis-server redis-6482.conf
redis-server redis-6483.conf
redis-server redis-6484.conf
redis-server redis-6485.conf
配置文件照你存放的路径启动
节点握手
当所有节点启动后,每个节点都是独立的,没有相互通信。这个时候需要通过**cluster meet {ip} {port}**命令建立通信。
[root@izwz9ja3exxb7by25st5gbz bin]# redis-cli -p 6480
127.0.0.1:6480> CLUSTER MEET 127.0.0.1 6481
127.0.0.1:6480> CLUSTER MEET 127.0.0.1 6482
127.0.0.1:6480> CLUSTER MEET 127.0.0.1 6483
127.0.0.1:6480> CLUSTER MEET 127.0.0.1 6484
127.0.0.1:6480> CLUSTER MEET 127.0.0.1 6485
这个时候6个节点已经建立起集群体系,但是集群还不是正常状态,发现state状态还是fail状态,因为集群的槽还没有被分配,可以看到cluster_slots_assigned=0。只有当16384个槽全部被分配后,集群才能正常使用。
127.0.0.1:6480> cluster info
cluster_state:fail
cluster_slots_assigned:0
cluster_slots_ok:0
分配槽
分别给6380,6381,6382三个主节点分配槽信息:
redis-cli -p 6480 cluster addslots {0..5461}
redis-cli -p 6481 cluster addslots {5462..10922}
redis-cli -p 6482 cluster addslots {10923..16383}
这个时候在查看集群状态信息,发现集群状态已为正常状态。
[root@izwz9ja3exxb7by25st5gbz bin]# redis-cli -p 6480
127.0.0.1:6480> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:6
cluster_stats_messages_sent:6118
cluster_stats_messages_received:6118
建立主从
经过上面的流程我们使用了6380,6381,6382三个节点,并分配了槽信息。还剩下三个节点作为从节点接入。使用cluster replicate {nodeId}命令建立主从信息。nodeId信息可以通过cluster nodes命令查看。
127.0.0.1:6480> CLUSTER NODES
ebfae9f86e63fbe9f16a298fc3a21d058be36113 127.0.0.1:6480 master - 0 0 7 connected 0-5461
2c121c8a45af8bc32e8265c3e00814277aeebcb8 127.0.0.1:6483 master
ccc3b4dadf1ad3afd9ffd374931d00bcdae4cae1 127.0.0.1:6485 slave
6bf1f8fb948ffa9fa08391516599f90971e45216 127.0.0.1:6481 master - 0 1617243345032 1 connected 5462-10922
9f745e96669a71876f74d4b3d6ce758f64dfcda8 127.0.0.1:6482 master - 0 1617243347036 2 connected 10923-16383
f59d6f10bf73e0090f4add04cc87fa154150880c 127.0.0.1:6484 master
//分别进入三个节点进行复制操作命令
127.0.0.1:6483> CLUSTER REPLICATE ebfae9f86e63fbe9f16a298fc3a21d058be36113
127.0.0.1:6484> CLUSTER REPLICATE 6bf1f8fb948ffa9fa08391516599f90971e45216
127.0.0.1:6485> CLUSTER REPLICATE 9f745e96669a71876f74d4b3d6ce758f64dfcda8
执行后在此查看集群节点信息:
127.0.0.1:6480> cluster nodes
ebfae9f86e63fbe9f16a298fc3a21d058be36113 127.0.0.1:6480 myself,master - 0 0 7 connected 0-5461
2c121c8a45af8bc32e8265c3e00814277aeebcb8 127.0.0.1:6483 slave ebfae9f86e63fbe9f16a298fc3a21d058be36113 0 1617243553451 7 connected
ccc3b4dadf1ad3afd9ffd374931d00bcdae4cae1 127.0.0.1:6485 slave 9f745e96669a71876f74d4b3d6ce758f64dfcda8 0 1617243549436 5 connected
6bf1f8fb948ffa9fa08391516599f90971e45216 127.0.0.1:6481 master - 0 1617243552449 1 connected 5462-10922
9f745e96669a71876f74d4b3d6ce758f64dfcda8 127.0.0.1:6482 master - 0 1617243551448 2 connected 10923-16383
f59d6f10bf73e0090f4add04cc87fa154150880c 127.0.0.1:6484 slave 6bf1f8fb948ffa9fa08391516599f90971e45216 0 1617243546432 1 connected
到这里集群就搭建完成,使用客户端测试一下:
[root@izwz9ja3exxb7by25st5gbz bin]# redis-cli -p 6480
127.0.0.1:6480> get layne
//发现layne key在6482节点上,会返回moved指令,而不是直接返回结果,需要自己转换节点去获取
(error) MOVED 12078 127.0.0.1:6482
127.0.0.1:6480> exit
//客户端连接加入 -c参数支持自动重定向,简化手动发起重定向操作。
[root@izwz9ja3exxb7by25st5gbz bin]# redis-cli -p 6480 -c
127.0.0.1:6480> get layne
-> Redirected to slot [12078] located at 127.0.0.1:6482
"redis"
redis-trib.rb
建立集群通信方式还有一种方式,就是redis-trib.rb。但是在redis 5以后可以直接使用redis-cli创建集群了。去redis官网可以看到介绍,原文如下:
Creating the cluster
Now that we have a number of instances running, we need to create our cluster by writing some meaningful configuration to the nodes.
If you are using Redis 5 or higher, this is very easy to accomplish as we are helped by the Redis Cluster command line utility embedded into redis-cli, that can be used to create new clusters, check or reshard an existing cluster, and so forth.
For Redis version 3 or 4, there is the older tool called redis-trib.rb which is very similar. You can find it in the src directory of the Redis source code distribution. You need to install redis gem to be able to run redis-trib.
To create your cluster for Redis 5 with redis-cli simply type:
redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 \
127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 \
--cluster-replicas 1
Using redis-trib.rb for Redis 4 or 3 type:
./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 \
127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005