Redis集群搭建

Redis集群搭建

1、准备工作

$ mkdir cluster-test
$ cd cluster-test
$ mkdir 7000 7001 7002 7003 7004 7005
$ cd 7000
$ touch redis.conf
$ vi redis.conf
# 添加如下集群配置文件
port 7000
cluster-enabled yes
cluster-config-file nodes_7000.conf
cluster-node-timeout 5000
appendonly yes
$ cp redis.conf ../7001
$ cp redis.conf ../7002
$ cp redis.conf ../7003
$ cp redis.conf ../7004
$ cp redis.conf ../7005
# 在文件夹 7000 至 7005 中，将各redis.conf 文件中的端口从 7000 改为与文件夹名字相同的号码，
# 并将 nodes_7001.conf 文件名的数字改为文件夹相同的号码。
$ vi ../7001/redis.conf
$ vi ../7002/redis.conf
$ vi ../7003/redis.conf
$ vi ../7004/redis.conf
$ vi ../7005/redis.conf
$ cat ../7001/redis.conf
daemonize    yes      		# redis后台运行
port 7001							# 端口7000,7002,7003
cluster-enabled yes   		# 开启集群
cluster-config-file nodes_7001.conf 		# 集群的配置  配置文件首次启动自动生成
cluster-node-timeout 5000  					# 请求超时 5s
appendonly yes										# aof日志开启  有需要就开启，它会每次写操作都记录一条日志

2、启动节点

​ 创建启动文件start_all.sh

./redis-server ~/cluster-test/7000/redis.conf
./redis-server ~/cluster-test/7001/redis.conf
./redis-server ~/cluster-test/7002/redis.conf 
./redis-server ~/cluster-test/7003/redis.conf 
./redis-server ~/cluster-test/7004/redis.conf 
./redis-server ~/cluster-test/7005/redis.conf 

​ 创建停止文件stop_all.sh

./redis-cli -p 7000 shutdown
./redis-cli -p 7001 shutdown
./redis-cli -p 7002 shutdown
./redis-cli -p 7003 shutdown
./redis-cli -p 7004 shutdown
./redis-cli -p 7005 shutdown

启动节点

./start_all.sh 执行启动文件

./start_all.sh 执行停止文件

3、创建集群

$ ./redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 127.0.0.1:7004 to 127.0.0.1:7000
Adding replica 127.0.0.1:7005 to 127.0.0.1:7001
Adding replica 127.0.0.1:7003 to 127.0.0.1:7002
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 7486f0b8cefd627fd6bdc8e779aca496ac8713bf 127.0.0.1:7000
   slots:[0-5460] (5461 slots) master
M: 979f7b82f0a3475d9a2c96d91986830558e961f6 127.0.0.1:7001
   slots:[5461-10922] (5462 slots) master
M: 6c52a1bcff1358b629e60a7bb929b49aa40a984d 127.0.0.1:7002
   slots:[10923-16383] (5461 slots) master
S: 2c784c3b98d5e0d02f22780e5d5c1c1f2222f75c 127.0.0.1:7003
   replicates 979f7b82f0a3475d9a2c96d91986830558e961f6
S: a0c47da88ce291186966a65d58138bfbc75d093d 127.0.0.1:7004
   replicates 6c52a1bcff1358b629e60a7bb929b49aa40a984d
S: 75e63be1f03b29ca2948339ee6cedc61c1a61596 127.0.0.1:7005
   replicates 7486f0b8cefd627fd6bdc8e779aca496ac8713bf
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
...
>>> Performing Cluster Check (using node 127.0.0.1:7000)
M: 7486f0b8cefd627fd6bdc8e779aca496ac8713bf 127.0.0.1:7000
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: 6c52a1bcff1358b629e60a7bb929b49aa40a984d 127.0.0.1:7002
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 75e63be1f03b29ca2948339ee6cedc61c1a61596 127.0.0.1:7005
   slots: (0 slots) slave
   replicates 7486f0b8cefd627fd6bdc8e779aca496ac8713bf
M: 979f7b82f0a3475d9a2c96d91986830558e961f6 127.0.0.1:7001
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: a0c47da88ce291186966a65d58138bfbc75d093d 127.0.0.1:7004
   slots: (0 slots) slave
   replicates 6c52a1bcff1358b629e60a7bb929b49aa40a984d
S: 2c784c3b98d5e0d02f22780e5d5c1c1f2222f75c 127.0.0.1:7003
   slots: (0 slots) slave
   replicates 979f7b82f0a3475d9a2c96d91986830558e961f6
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered. 
# 这表示集群中的 16384 个槽都有至少一个主节点在处理， 集群运作正常。

说明：–cluster-replicas 参数为数字，1表示每个主节点需要1个从节点。

4、测试

$ redis-cli -c -p 7000
127.0.0.1:7000> set foo bar
-> Redirected to slot [12182] located at 127.0.0.1:7002
OK
127.0.0.1:7002> set hello world
-> Redirected to slot [866] located at 127.0.0.1:7000
OK
127.0.0.1:7000> get foo
-> Redirected to slot [12182] located at 127.0.0.1:7002
"bar"
127.0.0.1:7002> get hello
-> Redirected to slot [866] located at 127.0.0.1:7000
"world"

查看集群信息和各个节点状态

$ redis-cli -p 7000 cluster nodes
6c52a1bcff1358b629e60a7bb929b49aa40a984d 127.0.0.1:7002@17002 master - 0 1620816059846 3 connected 10923-16383
75e63be1f03b29ca2948339ee6cedc61c1a61596 127.0.0.1:7005@17005 slave 7486f0b8cefd627fd6bdc8e779aca496ac8713bf 0 1620816058000 1 connected
979f7b82f0a3475d9a2c96d91986830558e961f6 127.0.0.1:7001@17001 master - 0 1620816058116 2 connected 5461-10922
7486f0b8cefd627fd6bdc8e779aca496ac8713bf 127.0.0.1:7000@17000 myself,master - 0 1620816058000 1 connected 0-5460
a0c47da88ce291186966a65d58138bfbc75d093d 127.0.0.1:7004@17004 slave 6c52a1bcff1358b629e60a7bb929b49aa40a984d 0 1620816059000 3 connected
2c784c3b98d5e0d02f22780e5d5c1c1f2222f75c 127.0.0.1:7003@17003 slave 979f7b82f0a3475d9a2c96d91986830558e961f6 0 1620816058000 2 connected
$ redis-cli -p 7000 cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:6801
cluster_stats_messages_pong_sent:6810
cluster_stats_messages_sent:13611
cluster_stats_messages_ping_received:6805
cluster_stats_messages_pong_received:6801
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:13611

添加一个新节点

redis-cli --cluster add-node 127.0.0.1:7006 127.0.0.1:7000

删除节点

redis-cli --cluster del-node 127.0.0.1:7000 `<node-id>`

问题：

$ redis-server ~/cluster-test/7000/redis.conf 
2397:C 12 May 2021 16:25:23.633 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
2397:C 12 May 2021 16:25:23.633 # Redis version=6.0.8, bits=64, commit=00000000, modified=0, pid=2397, just started
2397:C 12 May 2021 16:25:23.633 # Configuration loaded
2397:M 12 May 2021 16:25:23.635 * Increased maximum number of open files to 10032 (it was originally set to 256).
2397:M 12 May 2021 16:25:23.638 # Can't open the append-only file: Permission denied

解决：指令前加 sudo，暂时切换到超级用户模式以执行超级用户权限。

$ sudo redis-server ~/cluster-test/7001/redis.conf 
2683:C 12 May 2021 17:03:56.361 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
2683:C 12 May 2021 17:03:56.361 # Redis version=6.0.8, bits=64, commit=00000000, modified=0, pid=2683, just started
2683:C 12 May 2021 17:03:56.361 # Configuration loaded
2683:M 12 May 2021 17:03:56.365 * Increased maximum number of open files to 10032 (it was originally set to 2560).
2683:M 12 May 2021 17:03:56.367 # Sorry, the cluster configuration file nodes.conf is already used by a different Redis Cluster node. Please make sure that different nodes use different cluster configuration files.

解决：修改 redis.conf 文件中的 cluster-config-file nodes.conf 文件避免重名。

$ ./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005
/System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/universal-darwin19/rbconfig.rb:229: warning: Insecure world writable dir /Library/Frameworks/Python.framework/Versions/3.6/bin in PATH, mode 040777
WARNING: redis-trib.rb is not longer available!
You should use redis-cli instead.

All commands and features belonging to redis-trib.rb have been moved
to redis-cli.
In order to use them you should call redis-cli with the --cluster
option followed by the subcommand name, arguments and options.

Use the following syntax:
redis-cli --cluster SUBCOMMAND [ARGUMENTS] [OPTIONS]

Example:
redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 --cluster-replicas 1

To get help about all subcommands, type:
redis-cli --cluster help

解决：在redis5.0之后取消了ruby脚本 redis-trib.rb的支持（手动命令行添加集群的方式不变），创建集合到redis-cli里，避免了再安装ruby的相关环境。直接使用redis-clit的参数–cluster 来取代。

使用指令如下： ./redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 --cluster-replicas 1

Redis集群教程