06. Redis 环境搭建-分片集群Cluster

Redis 支持分片集群模式, 会将数据散落在不同节点之上, 且每个数据节点都有相应的副本集. 在Redis 3.x 时创建Redis 分片集群还比较麻烦, 需要借助于redis 源码中的ruby 脚本来创建，依赖ruby 环境。 而从Redis 5.x开始, , 创建Redis 分片集群不再支持ruby脚本创建, 而是使用redis-cli 命令直接创建。这种创建方式更为方便.

1. 集群配置文件模板

笔者将redis 集群配置存放在/usr/local/etc/redis 目录下

• 从redis源码目录下复制redis.conf 配置文件到/usr/local/etc/redis 目录下
• 创建集群配置模板文件: /usr/local/etc/redis/template-cluster.conf
• 新增节点时，只需要替换文件中的6379 即可.

# 引入原配置
include /usr/local/etc/redis/redis.conf

######################################## 自定义配置 ########################################

# 修改绑定ip
bind 127.0.0.1

# 设置关闭保护模式
protected-mode no

# 设置端口号为6379
port 6379

# 设置后台启动为yes
daemonize yes

# 设置pid文件
pidfile /logs/redis/redis.pid.6379

# 设置日志文件
logfile /logs/redis/redis.log.6379

# 设置RDB dum文件名
dbfilename dump.rdb.6379

# 设置dump文件存放目录
dir /data/redis

# 设置主节点密码
masterauth 123456

# 设置从节点只读
replica-read-only yes

# 设置当前节点访问密码为
requirepass 123456

# 设置客户端最多连接数
maxclients 1000

# 设置最大内存
maxmemory 2GB

# 设置达到最大内存后策略，报错
maxmemory-policy noeviction

# 启动AOF持久化模式
appendonly yes

# 设置AOF文件名
appendfilename "appendonly.aof.6379"

# 设置AOF写入磁盘策略, 每秒写入一次
appendfsync everysec

# 开启集群
cluster-enabled yes

# 集群节点连接超时时间
cluster-node-timeout 15000  

# 集群节点配置信息文件, 位于redis 数据文件存放目录，即$dir 定义的目录中
cluster-config-file nodes.conf.6379

2. 创建集群配置

对于Redis 分片集群, 笔者认为, 集群节点至少应该有9个。 三个主节点, 6个从节点. 这样集群才能集高可用和分片于一身. 笔者创建一个9个节点的集群, 端口号为：7481~7489

2.1 复制9个节点配置

配置文件均存放在/usr/local/etc/redis目录.

cp template-cluster.conf redis.conf.7481
cp template-cluster.conf redis.conf.7482
cp template-cluster.conf redis.conf.7483
cp template-cluster.conf redis.conf.7484
cp template-cluster.conf redis.conf.7485
cp template-cluster.conf redis.conf.7486
cp template-cluster.conf redis.conf.7487
cp template-cluster.conf redis.conf.7488
cp template-cluster.conf redis.conf.7489

2.2 替换端口号

依次替换每个节点配置的端口号，日志文件，dump文件等配置.

sed -i "s/6379/7481/g" redis.conf.7481
sed -i "s/6379/7482/g" redis.conf.7482
sed -i "s/6379/7483/g" redis.conf.7483
sed -i "s/6379/7484/g" redis.conf.7484
sed -i "s/6379/7485/g" redis.conf.7485
sed -i "s/6379/7486/g" redis.conf.7486
sed -i "s/6379/7487/g" redis.conf.7487
sed -i "s/6379/7488/g" redis.conf.7488
sed -i "s/6379/7489/g" redis.conf.7489

3. 启动redis 集群

3.1 启动redis

依次启动9台服务器!

redis-server redis.conf.7481
redis-server redis.conf.7482
redis-server redis.conf.7483
redis-server redis.conf.7484
redis-server redis.conf.7485
redis-server redis.conf.7486
redis-server redis.conf.7487
redis-server redis.conf.7488
redis-server redis.conf.7489

3.2 创建集群关系

使用redis-cli --cluster 命令建立集群关系， 需要注意的是，每个redis 都必须是空的, 即是新建的节点, 否则将无法创建。

$ redis-cli -a 123456 --cluster create \
> 127.0.0.1:7481 \
> 127.0.0.1:7482 \
> 127.0.0.1:7483 \
> 127.0.0.1:7484 \
> 127.0.0.1:7485 \
> 127.0.0.1:7486 \
> 127.0.0.1:7487 \
> 127.0.0.1:7488 \
> 127.0.0.1:7489 \
> --cluster-replicas 2


Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 9 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 127.0.0.1:7485 to 127.0.0.1:7481
Adding replica 127.0.0.1:7486 to 127.0.0.1:7481
Adding replica 127.0.0.1:7487 to 127.0.0.1:7482
Adding replica 127.0.0.1:7488 to 127.0.0.1:7482
Adding replica 127.0.0.1:7489 to 127.0.0.1:7483
Adding replica 127.0.0.1:7484 to 127.0.0.1:7483
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 9baa3a6f75fe188460d337cfa8d6df862412e6e4 127.0.0.1:7481
   slots:[0-5460] (5461 slots) master
M: 829f7c60a21a5033ce88f0c7e54b17083ca6a691 127.0.0.1:7482
   slots:[5461-10922] (5462 slots) master
M: 97324a77a0ebfa3ac237abfd2593347b93aad719 127.0.0.1:7483
   slots:[10923-16383] (5461 slots) master
S: 1362c9a5988eba9a227595670b3e98f70767935a 127.0.0.1:7484
   replicates 829f7c60a21a5033ce88f0c7e54b17083ca6a691
S: 7086538d6d4c11a5a52580082c4fd963814365cd 127.0.0.1:7485
   replicates 9baa3a6f75fe188460d337cfa8d6df862412e6e4
S: 3a5f4c13439bc20586238ef615e27a91dc47b251 127.0.0.1:7486
   replicates 9baa3a6f75fe188460d337cfa8d6df862412e6e4
S: c36937c4281e8f891e88b02102fe2f59c15d4769 127.0.0.1:7487
   replicates 97324a77a0ebfa3ac237abfd2593347b93aad719
S: 3865b037f0026306de024629a4b7298e9c80d4c9 127.0.0.1:7488
   replicates 97324a77a0ebfa3ac237abfd2593347b93aad719
S: 869576474847e228be3d9d5377b01dd3040b1cae 127.0.0.1:7489
   replicates 829f7c60a21a5033ce88f0c7e54b17083ca6a691
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
....
>>> Performing Cluster Check (using node 127.0.0.1:7481)
M: 9baa3a6f75fe188460d337cfa8d6df862412e6e4 127.0.0.1:7481
   slots:[0-5460] (5461 slots) master
   2 additional replica(s)
S: 7086538d6d4c11a5a52580082c4fd963814365cd 127.0.0.1:7485
   slots: (0 slots) slave
   replicates 9baa3a6f75fe188460d337cfa8d6df862412e6e4
S: c36937c4281e8f891e88b02102fe2f59c15d4769 127.0.0.1:7487
   slots: (0 slots) slave
   replicates 97324a77a0ebfa3ac237abfd2593347b93aad719
S: 3a5f4c13439bc20586238ef615e27a91dc47b251 127.0.0.1:7486
   slots: (0 slots) slave
   replicates 9baa3a6f75fe188460d337cfa8d6df862412e6e4
S: 3865b037f0026306de024629a4b7298e9c80d4c9 127.0.0.1:7488
   slots: (0 slots) slave
   replicates 97324a77a0ebfa3ac237abfd2593347b93aad719
M: 97324a77a0ebfa3ac237abfd2593347b93aad719 127.0.0.1:7483
   slots:[10923-16383] (5461 slots) master
   2 additional replica(s)
S: 869576474847e228be3d9d5377b01dd3040b1cae 127.0.0.1:7489
   slots: (0 slots) slave
   replicates 829f7c60a21a5033ce88f0c7e54b17083ca6a691
S: 1362c9a5988eba9a227595670b3e98f70767935a 127.0.0.1:7484
   slots: (0 slots) slave
   replicates 829f7c60a21a5033ce88f0c7e54b17083ca6a691
M: 829f7c60a21a5033ce88f0c7e54b17083ca6a691 127.0.0.1:7482
   slots:[5461-10922] (5462 slots) master
   2 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

3.3 查看集群状态

通过cluster nodes 命令, 可以看出redis各节点主从关系.

$ redis-cli -a 123456 -p 7481 cluster nodes
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
1c9ee4a2f7e0191983f67c8849e61f8cd5c91005 127.0.0.1:7482@17482 master - 0 1557385770000 2 connected 5461-10922
06993924bd468b6eddb20be332b762a956b6631b 127.0.0.1:7489@17489 slave 1f5f8ba346034004cdd52ef0f267f024b1be6ee3 0 1557385771602 9 connected
7229f95a77bb60065a78e3dc422c056e265b05b3 127.0.0.1:7485@17485 slave 57fb7a0de46ef6340ece8f6a67829c49aeb91c96 0 1557385769000 5 connected
e25d2f1a019a40282e3c9146a00e44f4aa077afa 127.0.0.1:7486@17486 slave 1c9ee4a2f7e0191983f67c8849e61f8cd5c91005 0 1557385770599 6 connected
1f5f8ba346034004cdd52ef0f267f024b1be6ee3 127.0.0.1:7483@17483 master - 0 1557385768000 3 connected 10923-16383
7464b7aafc58e9235fc2a5c40f999018bb8b5509 127.0.0.1:7484@17484 slave 57fb7a0de46ef6340ece8f6a67829c49aeb91c96 0 1557385768000 4 connected
fe1a76b8052de772ed3aa9472726507ee48a5f1d 127.0.0.1:7488@17488 slave 1c9ee4a2f7e0191983f67c8849e61f8cd5c91005 0 1557385769000 8 connected
57fb7a0de46ef6340ece8f6a67829c49aeb91c96 127.0.0.1:7481@17481 myself,master - 0 1557385769000 1 connected 0-5460
e72fb434274ce30e0da21130e9279c5a700419b6 127.0.0.1:7487@17487 slave 1f5f8ba346034004cdd52ef0f267f024b1be6ee3 0 1557385769596 7 connected