redis cluster mysql_redis cluster

redis cluster和小哨兵组合 高可用方案是一种全自动的办法

cluster 主要编辑3个参数

cluster-enabled yes #开启cluster，去掉注释

cluster-config-file nodes.conf

cluster-node-timeout 15000

启动server

[redis@redis02 ~]$ cat s.sh

pkill -9 redis

redis-server /u01/redis/redis-5.0.5/conf/7001/7001.conf

redis-server /u01/redis/redis-5.0.5/conf/7003/7003.conf

redis-server /u01/redis/redis-5.0.5/conf/7004/7004.conf

ps -ef|grep redis-server|grep -v grep

[redis@redis02 ~]$ sh s.sh

redis 15089 1 0 22:07 ? 00:00:00 redis-server 127.0.0.1:7001 [cluster]

redis 15094 1 0 22:07 ? 00:00:00 redis-server 127.0.0.1:7003 [cluster]

redis 15099 1 0 22:07 ? 00:00:00 redis-server 127.0.0.1:7004 [cluster]

创建cluster，redis5.0开始不再使用ruby搭建集群

[redis@redis01 ~]$ redis-cli --cluster create 192.168.20.201:7001 192.168.20.201:7002 192.168.20.202:7001

>>> Performing hash slots allocation on 3 nodes...

Master[0] -> Slots 0 - 5460

Master[1] -> Slots 5461 - 10922

Master[2] -> Slots 10923 - 16383

M: 797d123e6f8fb94260490b543cb8a5adce0d90db 192.168.20.201:7001

slots:[0-5460] (5461 slots) master

M: 4a3aff53bd451c3df2c497d576dad674fbf2ff26 192.168.20.201:7002

slots:[10923-16383] (5461 slots) master

M: fe3f033c4a899c032ac9c0696b89e7be7e4975b2 192.168.20.202:7001

slots:[5461-10922] (5462 slots) master

Can I set the above configuration? (type 'yes' to accept): yes

>>> Nodes configuration updated

>>> Assign a different config epoch to each node

>>> Sending CLUSTER MEET messages to join the cluster

Waiting for the cluster to join

...

>>> Performing Cluster Check (using node 192.168.20.201:7001)

M: 797d123e6f8fb94260490b543cb8a5adce0d90db 192.168.20.201:7001

slots:[0-5460] (5461 slots) master

M: fe3f033c4a899c032ac9c0696b89e7be7e4975b2 192.168.20.202:7001

slots:[5461-10922] (5462 slots) master

M: 4a3aff53bd451c3df2c497d576dad674fbf2ff26 192.168.20.201:7002

slots:[10923-16383] (5461 slots) master

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

查看cluster信息

[redis@redis01 ~]$ redis-cli -h 192.168.20.201 -p 7001 cluster info

cluster_state:ok

cluster_slots_assigned:16384

cluster_slots_ok:16384

cluster_slots_pfail:0

cluster_slots_fail:0

cluster_known_nodes:3

cluster_size:3

cluster_current_epoch:3

cluster_my_epoch:1

cluster_stats_messages_ping_sent:256

cluster_stats_messages_pong_sent:281

cluster_stats_messages_sent:537

cluster_stats_messages_ping_received:279

cluster_stats_messages_pong_received:256

cluster_stats_messages_meet_received:2

cluster_stats_messages_received:537

[redis@redis01 ~]$ redis-cli -h 192.168.20.201 -p 7001 cluster nodes

fe3f033c4a899c032ac9c0696b89e7be7e4975b2 192.168.20.202:7001@17001 master - 0 1568682616539 3 connected 5461-10922

4a3aff53bd451c3df2c497d576dad674fbf2ff26 192.168.20.201:7002@17002 master - 0 1568682618047 2 connected 10923-16383

797d123e6f8fb94260490b543cb8a5adce0d90db 192.168.20.201:7001@17001 myself,master - 0 1568682615000 1 connected 0-5460

添加从节点

[redis@redis01 ~]$ redis-cli --cluster add-node 192.168.20.201:7003 192.168.20.202:7001 --cluster-slave

>>> Adding node 192.168.20.201:7003 to cluster 192.168.20.202:7001

>>> Performing Cluster Check (using node 192.168.20.202:7001)

M: fe3f033c4a899c032ac9c0696b89e7be7e4975b2 192.168.20.202:7001

slots:[5461-10922] (5462 slots) master

M: 4a3aff53bd451c3df2c497d576dad674fbf2ff26 192.168.20.201:7002

slots:[10923-16383] (5461 slots) master

M: 797d123e6f8fb94260490b543cb8a5adce0d90db 192.168.20.201:7001

slots:[0-5460] (5461 slots) master

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

Automatically selected master 192.168.20.202:7001

>>> Send CLUSTER MEET to node 192.168.20.201:7003 to make it join the cluster.

Waiting for the cluster to join

查看nodes

[redis@redis01 ~]$ redis-cli -h 192.168.20.201 -p 7001 cluster nodes

4a3aff53bd451c3df2c497d576dad674fbf2ff26 192.168.20.201:7002@17002 master - 0 1568712829454 2 connected 10923-16383

4464a46e8f73931feea8991c6ce9bde7277c41aa 192.168.20.202:7003@17003 slave 797d123e6f8fb94260490b543cb8a5adce0d90db 0 1568712831825 1 connected

fe3f033c4a899c032ac9c0696b89e7be7e4975b2 192.168.20.202:7001@17001 master - 0 1568712832857 3 connected 5461-10922

7fa3dc96bf77976e0612aa55f872ddb3c9f271f1 192.168.20.202:7004@17004 slave 4a3aff53bd451c3df2c497d576dad674fbf2ff26 0 1568712831000 2 connected

797d123e6f8fb94260490b543cb8a5adce0d90db 192.168.20.201:7001@17001 myself,master - 0 1568712829000 1 connected 0-5460

39336bfdf1b0c35435ecebe66afb6f79e78339ba 192.168.20.201:7003@17003 slave fe3f033c4a899c032ac9c0696b89e7be7e4975b2 0 1568712830537 3 connected

添加master

[redis@redis01 ~]$ redis-cli --cluster add-node 192.168.20.201:7003 192.168.20.202:7001

>>> Adding node 192.168.20.201:7003 to cluster 192.168.20.202:7001

>>> Performing Cluster Check (using node 192.168.20.202:7001)

M: fe3f033c4a899c032ac9c0696b89e7be7e4975b2 192.168.20.202:7001

slots:[5461-10922] (5462 slots) master

M: 4a3aff53bd451c3df2c497d576dad674fbf2ff26 192.168.20.201:7002

slots:[10923-16383] (5461 slots) master

M: 797d123e6f8fb94260490b543cb8a5adce0d90db 192.168.20.201:7001

slots:[0-5460] (5461 slots) master

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

>>> Send CLUSTER MEET to node 192.168.20.201:7003 to make it join the cluster.

[OK] New node added correctly.

[redis@redis01 ~]$ redis-cli -h 192.168.20.201 -p 7002 cluster nodes

fe3f033c4a899c032ac9c0696b89e7be7e4975b2 192.168.20.202:7001@17001 master - 0 1568711371227 3 connected 5461-10922

046582928a7b3d21aa2a344d01f182c489e7f26d 192.168.20.201:7003@17003 master - 0 1568711370000 0 connected

797d123e6f8fb94260490b543cb8a5adce0d90db 192.168.20.201:7001@17001 master - 0 1568711372564 1 connected 0-5460

4a3aff53bd451c3df2c497d576dad674fbf2ff26 192.168.20.201:7002@17002 myself,master - 0 1568711371000 2 connected 10923-16383

删除节点 和 删除复制节点命令是一样的

[redis@redis02 ~]$ redis-cli --cluster del-node 192.168.20.201:7003 100a5f359104a7e13e1bccd02677032cfab667ea

>>> Removing node 100a5f359104a7e13e1bccd02677032cfab667ea from cluster 192.168.20.201:7003

>>> Sending CLUSTER FORGET messages to the cluster...

>>> SHUTDOWN the node.

检查节点

[redis@redis01 ~]$ redis-cli --cluster check 192.168.20.201:7002

Could not connect to Redis at 192.168.20.201:7001: Connection refused

192.168.20.201:7002 (4a3aff53...) -> 1 keys | 5461 slots | 1 slaves.

192.168.20.202:7001 (fe3f033c...) -> 1 keys | 5462 slots | 1 slaves.

192.168.20.202:7003 (4464a46e...) -> 0 keys | 5461 slots | 0 slaves.

[OK] 2 keys in 3 masters.

0.00 keys per slot on average.

>>> Performing Cluster Check (using node 192.168.20.201:7002)

M: 4a3aff53bd451c3df2c497d576dad674fbf2ff26 192.168.20.201:7002

slots:[10923-16383] (5461 slots) master

1 additional replica(s)

M: fe3f033c4a899c032ac9c0696b89e7be7e4975b2 192.168.20.202:7001

slots:[5461-10922] (5462 slots) master

1 additional replica(s)

S: 39336bfdf1b0c35435ecebe66afb6f79e78339ba 192.168.20.201:7003

slots: (0 slots) slave

replicates fe3f033c4a899c032ac9c0696b89e7be7e4975b2

S: 7fa3dc96bf77976e0612aa55f872ddb3c9f271f1 192.168.20.202:7004

slots: (0 slots) slave

replicates 4a3aff53bd451c3df2c497d576dad674fbf2ff26

M: 4464a46e8f73931feea8991c6ce9bde7277c41aa 192.168.20.202:7003

slots:[0-5460] (5461 slots) master

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

搭建cluster过程当中遇到的错误

错误1

[redis@redis01 redis-5.0.5]$ make install

cd src && make install

make[1]: Entering directory `/u01/redis/redis-5.0.5/src'

Hint: It's a good idea to run 'make test' ;)

INSTALL install

install: cannot create regular file `/usr/local/bin/redis-server': Permission denied

make[1]: *** [install] Error 1

make[1]: Leaving directory `/u01/redis/redis-5.0.5/src'

make: *** [install] Error 2

make install 使用root用户执行

错误2

[redis@redis01 ~]$ redis-cli --cluster create 192.168.20.201:7001 192.168.20.201:7002 192.168.20.202:7001

[ERR] Node 192.168.20.201:7001 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.

[redis@redis01 ~]$ redis-cli -h 192.168.20.201 -p 7001 CLUSTER NODES

571fc5a63b009d201c50a90d45fa23452d43d596 127.0.0.1:7001@17001 myself,master - 0 1568642020827 1 connected 0-4095 [7365-

0f5ee2036d799381612b76f423072d3760463697 :0@0 master,fail?,noaddr - 1568642020835 1568642020827 3 disconnected 8192-12287

70ff508977ef8bd4422d073dd5f79cefc164d193 127.0.0.1:7004@17004 master,fail? - 1568642020835 1568642020827 4 disconnected 12288-16383

f3f3d854bce9e15955bfcfe78d71c943025d7b30 :0@0 master,fail?,noaddr - 1568642020835 1568642020827 2 disconnected 4096-8191

[redis@redis01 ~]$ sh s.sh

redis 14997 1 0 22:16 ? 00:00:00 redis-server /u01/redis/redis-5.0.5/conf/7001/7001.conf

redis 14999 1 0 22:16 ? 00:00:00 redis-server /u01/redis/redis-5.0.5/conf/7002/7002.conf

redis 15001 1 0 22:16 ? 00:00:00 redis-server /u01/redis/redis-5.0.5/conf/7003/7003.conf

[redis@redis01 ~]$ redis-cli -h 192.168.20.201 -p 7001 CLUSTER NODES

13f63e9b5a7696181f7ae06f7ffc08b3afbb5cc3 :7001@17001 myself,master - 0 0 0 connected 3168 3300 7365 11298 15363 15495

将每个节点下aof、rdb、nodes.conf本地备份文件删除

flushdb

错误3

Waiting for the cluster to join hang

修改conf文件中 bind 修改成真实的ip