linux在6台主机搭建3组一主两从的redis哨兵集群
序言
之前在测试环境搭建了3组一主两从的redis集群,最近又在生产环境成功搭建,为了便于以后自己查看,所以也做一下记录。
其实也可以当成搭建redis集群,搭建完成之后改一下配置就可以了。
我之前写了一篇搭建reids集群的博客:https://blog.csdn.net/fhf2424045058/article/details/105430060可供参考!
不过我还是要从头再来!!!
部署主机情况说明
我们的生产环境这次只给我分了6台主机,主机有限,但也不是不够,所以依然和测试一样搭建9个节点。
如果主机比较多,可以直接来它个12台,这feel真好!
说明:下表中的IP地址,在生产环境肯定不是如此的,为了保密,所以用127来代替。
主机列表和端口分布
主机IP | 端口 |
---|---|
127.0.0.1 | 7000、7008 |
127.0.0.2 | 7001 |
127.0.0.3 | 7002、7003 |
127.0.0.4 | 70004 |
127.0.0.5 | 7005、7006 |
127.0.0.6 | 7007 |
其中7000、7003和7006是master,其他的是slave,放上我写的我们生产的部署情况的一点点截图来说明一下关系
下面就开始从头搭建啦~
一、redis一主两从集群搭建
1、新建用户
在6台主机新建用户
groupadd rdsclsr
useradd -m -d /bal/rdsclsr -g rdsclsr -s /bin/bash -u 501 rdsclsr
2、解压
在6台主机解压redis安装包
tar -zxvf redis-5.0.8.tar.gz
3、编译redis
cd /bal/rdsclsr/redis/redis-5.0.8/
mkdir installs
make
4、安装redis
还是老习惯,安装到普通用户下的自定义的目录
make PREFIX=/bal/rdsclsr/redis/redis-5.0.8/installs install
5、创建节点目录
在不同主机创建9个redis节点配置文件目录
从这里开始:我就不详细再列出是哪台主机了,你一定可以根据我列出的命令来判断是在哪台主机。
cd /bal/rdsclsr/redis/
mkdir -p rediscluster/s_1_1_7000/data
mkdir -p rediscluster/s_1_2_7001/data
mkdir -p rediscluster/s_1_3_7002/data
mkdir -p rediscluster/s_2_1_7003/data
mkdir -p rediscluster/s_2_2_7004/data
mkdir -p rediscluster/s_2_3_7005/data
mkdir -p rediscluster/s_3_1_7006/data
mkdir -p rediscluster/s_3_2_7007/data
mkdir -p rediscluster/s_3_3_7008/data
说明:可能有小伙伴看到这里就会疑问,为啥端口前面要加上s_1_1这样的,当然是为了方便运维,当你打开的主机过多的时候,只看端口会一时反映不过来,这是哪组的主还是备,当然也可以根据自己的喜好来创建。
6、修改配置文件(见附录一)
修改6台主机中redis的配置文件
vim /bal/rdsclsr/redis/rediscluster/s_1_1_7000/redis-7000.conf
vim /bal/rdsclsr/redis/rediscluster/s_1_2_7001/redis-7001.conf
vim /bal/rdsclsr/redis/rediscluster/s_1_3_7002/redis-7002.conf
vim /bal/rdsclsr/redis/rediscluster/s_2_1_7003/redis-7003.conf
vim /bal/rdsclsr/redis/rediscluster/s_2_2_7004/redis-7004.conf
vim /bal/rdsclsr/redis/rediscluster/s_2_3_7005/redis-7005.conf
vim /bal/rdsclsr/redis/rediscluster/s_3_1_7006/redis-7006.conf
vim /bal/rdsclsr/redis/rediscluster/s_3_2_7007/redis-7007.conf
vim /bal/rdsclsr/redis/rediscluster/s_3_3_7008/redis-7008.conf
说明:配置文件Master和slave的配置文件内容和说明我会粘贴在本文的最后,可以直接跳到最后看(左侧有目录哦~)。
7、启动9个redis节点
cd /bal/rdsclsr/redis/redis-5.0.8/installs/bin
./redis-server /bal/rdsclsr/redis/rediscluster/s_1_1_7000/redis-7000.conf
./redis-server /bal/rdsclsr/redis/rediscluster/s_1_2_7001/redis-7001.conf
./redis-server /bal/rdsclsr/redis/rediscluster/s_1_3_7002/redis-7002.conf
./redis-server /bal/rdsclsr/redis/rediscluster/s_2_1_7003/redis-7003.conf
./redis-server /bal/rdsclsr/redis/rediscluster/s_2_2_7004/redis-7004.conf
./redis-server /bal/rdsclsr/redis/rediscluster/s_2_3_7005/redis-7005.conf
./redis-server /bal/rdsclsr/redis/rediscluster/s_3_1_7006/redis-7006.conf
./redis-server /bal/rdsclsr/redis/rediscluster/s_3_2_7007/redis-7007.conf
./redis-server /bal/rdsclsr/redis/rediscluster/s_3_3_7008/redis-7008.conf
8、编写9个节点的启动脚本
(1)创建脚本目录
cd /bal/rdsclsr/redis/rediscluster
mkdir updownsh
(2)编辑脚本文件
vim /bal/rdsclsr/redis/rediscluster/updownsh/up_s_1_1_7000.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/up_s_1_2_7001.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/up_s_1_3_7002.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/up_s_2_1_7003.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/up_s_2_2_7004.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/up_s_2_3_7005.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/up_s_3_1_7006.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/up_s_3_2_7007.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/up_s_3_3_7008.sh
(3)9个节点启动脚本内容:
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-server /bal/rdsclsr/redis/rediscluster/s_1_1_7000/redis-7000.conf
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-server /bal/rdsclsr/redis/rediscluster/s_1_2_7001/redis-7001.conf
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-server /bal/rdsclsr/redis/rediscluster/s_1_3_7002/redis-7002.conf
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-server /bal/rdsclsr/redis/rediscluster/s_2_1_7003/redis-7003.conf
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-server /bal/rdsclsr/redis/rediscluster/s_2_2_7004/redis-7004.conf
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-server /bal/rdsclsr/redis/rediscluster/s_2_3_7005/redis-7005.conf
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-server /bal/rdsclsr/redis/rediscluster/s_3_1_7006/redis-7006.conf
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-server /bal/rdsclsr/redis/rediscluster/s_3_2_7007/redis-7007.conf
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-server /bal/rdsclsr/redis/rediscluster/s_3_3_7008/redis-7008.conf
说明:从这里和后面的脚本中:9个脚本以#!/bin/bash
开头为一个脚本,我也就不分开一行一行的写在这里了,在编辑的时候以#!/bin/bash
为分割是一个脚本。
(4)给9个启动脚本赋予执行权限
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/up_s_1_1_7000.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/up_s_1_2_7001.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/up_s_1_3_7002.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/up_s_2_1_7003.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/up_s_2_2_7004.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/up_s_2_3_7005.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/up_s_3_1_7006.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/up_s_3_2_7007.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/up_s_3_3_7008.sh
9、编写9个节点的停止脚本
(1)编辑脚本文件
vim /bal/rdsclsr/redis/rediscluster/updownsh/down_s_1_1_7000.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/down_s_1_2_7001.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/down_s_1_3_7002.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/down_s_2_1_7003.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/down_s_2_2_7004.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/down_s_2_3_7005.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/down_s_3_1_7006.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/down_s_3_2_7007.sh
vim /bal/rdsclsr/redis/rediscluster/updownsh/down_s_3_3_7008.sh
(2)9个节点停止脚本内容:
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.1 -p 7000 shutdown
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.2 -p 7001 shutdown
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.3 -p 7002 shutdown
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.3 -p 7003 shutdown
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.4 -p 7004 shutdown
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.5 -p 7005 shutdown
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.5 -p 7006 shutdown
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.6 -p 7007 shutdown
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.1 -p 7008 shutdown
(3)给9个停止脚本赋予执行权限
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/down_s_1_1_7000.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/down_s_1_2_7001.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/down_s_1_3_7002.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/down_s_2_1_7003.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/down_s_2_2_7004.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/down_s_2_3_7005.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/down_s_3_1_7006.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/down_s_3_2_7007.sh
chmod +x /bal/rdsclsr/redis/rediscluster/updownsh/down_s_3_3_7008.sh
10、编辑登录客户端脚本
(1)脚本编辑命令
cd /bal/rdsclsr/redis/rediscluster/updownsh
vim redis_cli_s_1_1_7000.sh
vim redis_cli_s_1_2_7001.sh
vim redis_cli_s_1_3_7002.sh
vim redis_cli_s_2_1_7003.sh
vim redis_cli_s_2_2_7004.sh
vim redis_cli_s_2_3_7005.sh
vim redis_cli_s_3_1_7006.sh
vim redis_cli_s_3_2_7007.sh
vim redis_cli_s_3_3_7008.sh
(2)登录客户端命令
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7000
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7001
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7002
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7003
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7004
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7005
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7006
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7007
#!/bin/bash
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7008
(3)赋予执行权限
chmod +x redis_cli_s_1_1_7000.sh
chmod +x redis_cli_s_1_2_7001.sh
chmod +x redis_cli_s_1_3_7002.sh
chmod +x redis_cli_s_2_1_7003.sh
chmod +x redis_cli_s_2_2_7004.sh
chmod +x redis_cli_s_2_3_7005.sh
chmod +x redis_cli_s_3_1_7006.sh
chmod +x redis_cli_s_3_2_7007.sh
chmod +x redis_cli_s_3_3_7008.sh
二、Redis哨兵部署
3个master端口和主机搭建和配置哨兵
主机IP | 端口-角色 |
---|---|
127.0.0.1 | 7000-master |
127.0.0.3 | 7003-master |
127.0.0.5 | 7006-master |
1、创建sentinel目录
在3组哨兵的主机中创建sentinel的配置文件目录和日志目录
cd /bal/rdsclsr/redis
mkdir sentinel
2、修改哨兵的配置文件(见附录二)
cp /bal/rdsclsr/redis/redis-5.0.8/sentinel.conf /bal/rdsclsr/redis/sentinel
vim /bal/rdsclsr/redis/sentinel/sentinel.conf
说明:可以看左侧的目录直接找到附录二哦~
3、在3台master主机启动sentinel
(1)编辑启动脚本
cd /bal/rdsclsr/redis/sentinel
vim upsentinel.sh
sentinel启动脚本内容
#!/bin/sh
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-sentinel /bal/rdsclsr/redis/sentinel/sentinel.conf
赋予执行权限并启动
chmod +x upsentinel.sh
sh upsentinel.sh
(2)编辑停止脚本
cd /bal/rdsclsr/redis/sentinel
vim downsentinel.sh
3个sentinel的停止脚本内容
#!/bin/sh
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.1 -p 25379 shutdown
#!/bin/sh
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.3 -p 25379 shutdown
#!/bin/sh
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -h 127.0.0.5 -p 25379 shutdown
赋予执行权限
chmod +x downsentinel.sh
三、查看主从信息
------------------------------------这里以7000这一组为例---------------------------------------
查看master所在主机的进程:
ps -ef|grep redis
显示内容:
rdsclsr 28017 1 0 14:40 ? 00:00:02 /bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-sentinel 127.0.0.1:25379 [sentinel]
rdsclsr 31013 1 0 15:28 ? 00:00:00 ./redis-server *:7000
rdsclsr 32555 26320 0 15:53 pts/1 00:00:00 grep --color=auto redis
(1)查看第一组redis哨兵的master
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7000 info replication
显示内容:
# Replication
role:master
connected_slaves:2
slave0:ip=127.0.0.2,port=7001,state=online,offset=6412164,lag=0
slave1:ip=127.0.0.3,port=7002,state=online,offset=6412164,lag=0
master_replid:6008be2704b2e007b93d784613e41d205027f128
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:6412164
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:5363589
repl_backlog_histlen:1048576
(2)查看第一组redis哨兵slave1
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7001 info replication
显示内容:
# Replication
role:slave
master_host:127.0.0.1
master_port:7000
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:6418476
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:6008be2704b2e007b93d784613e41d205027f128
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:6418476
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:5369901
repl_backlog_histlen:1048576
(3)查看第一组redis哨兵slave2
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7002 info replication
显示内容:
# Replication
role:slave
master_host:127.0.0.1
master_port:7000
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:6418955
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:6008be2704b2e007b93d784613e41d205027f128
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:6418955
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:5370380
repl_backlog_histlen:1048576
(4)模拟master宕机,选举slave2为master
/bal/rdsclsr/redis/redis-5.0.8/installs/bin/redis-cli -p 7002 info replication
显示内容:
# Replication
role:master
connected_slaves:1
slave0:ip=127.0.0.2,port=7001,state=online,offset=277553,lag=0
master_replid:ccfd0f854252af7c7c3ae2da03a0a0ea143596ad
master_replid2:d6a8f4c9052a3169d6ea8f54c76a9b9169cd31fa
master_repl_offset:277553
second_repl_offset:271060
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:277553
到这里三组哨兵,每组一主两备就搭建完成了。
最后附上附录一和附录二
附录一
为了不让篇幅太长,我就把里面的注释去掉
protected-mode no
port 7000
tcp-backlog 511
timeout 300
tcp-keepalive 300
daemonize yes
supervised no
pidfile "/bal/rdsclsr/redis/rediscluster/s_1_1_7000/redis_s_1_1_7000.pid"
loglevel notice
logfile "/bal/rdsclsr/redis/rediscluster/s_1_1_7000/redis_s_1_1_7000.log"
databases 16
always-show-logo yes
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename "dump.s_1_1_7000.rdb"
dir "/bal/rdsclsr/redis/rediscluster/s_1_1_7000/data"
################################# REPLICATION #################################
# replicaof <masterip> <masterport>
################################# REPLICATION #################################
replica-serve-stale-data yes
replica-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
replica-priority 100
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
stream-node-max-bytes 4096
stream-node-max-entries 100
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
dynamic-hz yes
aof-rewrite-incremental-fsync yes
rdb-save-incremental-fsync yes
maxclients 4064
说明1 : redis.conf的配置项比较多,但如果要改成其他的节点的配置,需要修改的内容也并不多,
其他两组redis的master只需修改port、 pidfile、 logfile、 dbfilename、dir这5个值即可。
说明2:我的附录一是以7000为例的,而6个slave的redis.conf配置文件和master的redis.conf配置文件除了port、 pidfile、 logfile、 dbfilename、dir这5个值不同以外,
7000的2个slave的配置文件还需要添加 ”replicaof 127.0.0.1 7000
” 配置,在这个附录一我特意留了几行注释:
################################# REPLICATION #################################
replicaof <masterip> <masterport>
################################# REPLICATION #################################
把里面的masterip和masterport换成master的ip和port即可,也就是slave需要添加对应的master的主机IP和端口。
所以,其他的5个slave的redis.conf的配置文件需要就只需要修改port、 pidfile、 logfile、 dbfilename、dir和replicaof 即可。
说明3:如果有时间和兴趣可以把注释都翻译一下,或者用翻译工具也可以,配置项的解释可以在网上找找,如果不想找可以看我之前参考的博客:文章链接
附录二
# sentinel configuration by fanhf in 202006152012
port 25379
daemonize yes
pidfile "/bal/rdsclsr/redis/sentinel/redis-sentinel.pid"
logfile "/bal/rdsclsr/redis/sentinel/redis-sentinel.log"
dir "/bal/rdsclsr/redis/sentinel"
protected-mode no
sentinel deny-scripts-reconfig no
sentinel monitor redis_22122_group1 127.0.0.1 7000 1
sentinel down-after-milliseconds redis_22122_group1 10000
sentinel failover-timeout redis_22122_group1 10000
sentinel parallel-syncs redis_22122_group1 1
sentinel config-epoch redis_22122_group1 0
sentinel monitor redis_22122_group2 127.0.0.3 7003 1
sentinel down-after-milliseconds redis_22122_group2 10000
sentinel failover-timeout redis_22122_group2 10000
sentinel parallel-syncs redis_22122_group2 1
sentinel config-epoch redis_22122_group2 0
sentinel monitor redis_22122_group3 127.0.0.5 7006 1
sentinel down-after-milliseconds redis_22122_group3 10000
sentinel failover-timeout redis_22122_group3 10000
sentinel parallel-syncs redis_22122_group3 1
sentinel config-epoch redis_22122_group3 0
说明:在3台master所在主机同时还有sentinel,每个sentine的配置都是一样的,也就是说每个sentinel都会监控9个节点。
到这里就搭建完成了,在搭建的过程中有任何问题可以留言,相互交流,共同进步!
与君共勉!!!