文章目录
前言
Redis Sentinel(哨兵)是官网推荐的redis高可用解决方案。Sentinel的稳定版自Redis 2.8开始正式发布。
一、基于Sentinel的Redis HA架构
常见Redis HA结构,如上图,包含一主、两从、三哨兵,是Reids主从模式+哨兵集群的组合。其中:
- Redis Master(主节点):负责处理redis读、写请求;
- Redis Slave(从节点):负责从redis主节点同步数据,保持与主节点的数据同步;处理redis读请求;
- Sentinel(哨兵):负责监管整个redis主从结构;
二、搭建步骤
2.1 Redis 源码编译
从 https://redis.io/download 下载最新版 Redis 6.0.9,需要自行编译。下载解压后,按照官网提示的安装步骤进行操作,对源码进行编译。
2.2 Redis主从结构
编译完成后,进入redis-6.0.9解压、编译目录,将配置文件redis.conf复制三份,分别命名、修改以下配置。
-
redis-master.conf 主节点配置
#...... # Accept connections on the specified port, default is 6379 (IANA #815344). # If port 0 is specified Redis will not listen on a TCP socket. port 6380 # If a pid file is specified, Redis writes it where specified at startup # and removes it at exit. # # When the server runs non daemonized, no pid file is created if none is # specified in the configuration. When the server is daemonized, the pid file # is used even if not specified, defaulting to "/var/run/redis.pid". # # Creating a pid file is best effort: if Redis is not able to create it # nothing bad happens, the server will start and run normally. pidfile /var/run/redis_6380.pid
-
redis-slave1.conf,从节点1配置,将主节点设置为6380。
#...... # Accept connections on the specified port, default is 6379 (IANA #815344). # If port 0 is specified Redis will not listen on a TCP socket. port 6381 # If a pid file is specified, Redis writes it where specified at startup # and removes it at exit. # # When the server runs non daemonized, no pid file is created if none is # specified in the configuration. When the server is daemonized, the pid file # is used even if not specified, defaulting to "/var/run/redis.pid". # # Creating a pid file is best effort: if Redis is not able to create it # nothing bad happens, the server will start and run normally. pidfile /var/run/redis_6381.pid # Master-Replica replication. Use replicaof to make a Redis instance a copy of # another Redis server. A few things to understand ASAP about Redis replication. # # +------------------+ +---------------+ # | Master | ---> | Replica | # | (receive writes) | | (exact copy) | # +------------------+ +---------------+ # # 1) Redis replication is asynchronous, but you can configure a master to # stop accepting writes if it appears to be not connected with at least # a given number of replicas. # 2) Redis replicas are able to perform a partial resynchronization with the # master if the replication link is lost for a relatively small amount of # time. You may want to configure the replication backlog size (see the next # sections of this file) with a sensible value depending on your needs. # 3) Replication is automatic and does not need user intervention. After a # network partition replicas automatically try to reconnect to masters # and resynchronize with them. # replicaof 127.0.0.1 6380
-
redis-slave2.conf,从节点2配置,将主节点设置为6380。
port 6382 pidfile /var/run/redis_6382.pid replicaof 127.0.0.1 6380
在redis-6.0.9目录下,依次通过下述命令来启动redis节点。
src/redis-server redis-master.conf
src/redis-server redis-slave1.conf
从redis的启动日志中,可以看到redis从节点连接到主节点、同步数据的相关情况。
2.3 Redis Sentinels
在redis-6.0.9目录下,将sentinel.conf配置复制三份,分别重命名、修改以下配置。
-
sentinel_26380.conf
# *** IMPORTANT *** # # By default Sentinel will not be reachable from interfaces different than # localhost, either use the 'bind' directive to bind to a list of network # interfaces, or disable protected mode with "protected-mode no" by # adding it to this configuration file. # # Before doing that MAKE SURE the instance is protected from the outside # world via firewalling or other means. # # For example you may use one of the following: # # bind 127.0.0.1 192.168.1.1 # # protected-mode no # port <sentinel-port> # The port that this sentinel instance will run on port 26380 # When running daemonized, Redis Sentinel writes a pid file in # /var/run/redis-sentinel.pid by default. You can specify a custom pid file # location here. pidfile /var/run/redis-sentinel-26380.pid # sentinel monitor <master-name> <ip> <redis-port> <quorum> # # Tells Sentinel to monitor this master, and to consider it in O_DOWN # (Objectively Down) state only if at least <quorum> sentinels agree. # # Note that whatever is the ODOWN quorum, a Sentinel will require to # be elected by the majority of the known Sentinels in order to # start a failover, so no failover can be performed in minority. # # Replicas are auto-discovered, so you don't need to specify replicas in # any way. Sentinel itself will rewrite this configuration file adding # the replicas using additional configuration options. # Also note that the configuration file is rewritten when a # replica is promoted to master. # # Note: master name should not include special characters or spaces. # The valid charset is A-z 0-9 and the three characters ".-_". sentinel monitor mymaster 127.0.0.1 6380 2
告诉当前Sentinel,要监视的redis主节点为:127.0.0.1 6380。
通过src/redis-sentinel sentinel_26380.conf 来启动Sentinel。
> src/redis-sentinel sentinel_26380.conf 1043:X 15 Nov 2020 10:59:52.528 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 1043:X 15 Nov 2020 10:59:52.528 # Redis version=6.0.9, bits=64, commit=00000000, modified=0, pid=1043, just started 1043:X 15 Nov 2020 10:59:52.528 # Configuration loaded 1043:X 15 Nov 2020 10:59:52.529 * Increased maximum number of open files to 10032 (it was originally set to 2560). _._ _.-``__ ''-._ _.-`` `. `_. ''-._ Redis 6.0.9 (00000000/0) 64 bit .-`` .-```. ```\/ _.,_ ''-._ ( ' , .-` | `, ) Running in sentinel mode |`-._`-...-` __...-.``-._|'` _.-'| Port: 26380 | `-._ `._ / _.-' | PID: 1043 `-._ `-._ `-./ _.-' _.-' |`-._`-._ `-.__.-' _.-'_.-'| | `-._`-._ _.-'_.-' | http://redis.io `-._ `-._`-.__.-'_.-' _.-' |`-._`-._ `-.__.-' _.-'_.-'| | `-._`-._ _.-'_.-' | `-._ `-._`-.__.-'_.-' _.-' `-._ `-.__.-' _.-' `-._ _.-' `-.__.-' 1043:X 15 Nov 2020 10:59:52.531 # Sentinel ID is 294d622af53ec541a775385baf7cdf99a34f3163 1043:X 15 Nov 2020 10:59:52.531 # +monitor master mymaster 127.0.0.1 6380 quorum 2 1043:X 15 Nov 2020 10:59:52.532 * +slave slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6380 1043:X 15 Nov 2020 10:59:52.533 * +slave slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6380
从日志中可以看到,虽然Sentinel配置文件中仅设置了redis主节点的ip、端口,但Sentinel仍然能通过主节点来获取其从节点(6381、6382)的信息。
-
sentinel_26381.conf
port 26381 pidfile /var/run/redis-sentinel-26381.pid # 监视同一主节点 sentinel monitor mymaster 127.0.0.1 6380 2
启动Sentinel。
> src/redis-sentinel sentinel_26381.conf 1089:X 15 Nov 2020 11:09:41.782 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 1089:X 15 Nov 2020 11:09:41.782 # Redis version=6.0.9, bits=64, commit=00000000, modified=0, pid=1089, just started 1089:X 15 Nov 2020 11:09:41.782 # Configuration loaded 1089:X 15 Nov 2020 11:09:41.783 * Increased maximum number of open files to 10032 (it was originally set to 2560). _._ _.-``__ ''-._ _.-`` `. `_. ''-._ Redis 6.0.9 (00000000/0) 64 bit .-`` .-```. ```\/ _.,_ ''-._ ( ' , .-` | `, ) Running in sentinel mode |`-._`-...-` __...-.``-._|'` _.-'| Port: 26381 | `-._ `._ / _.-' | PID: 1089 `-._ `-._ `-./ _.-' _.-' |`-._`-._ `-.__.-' _.-'_.-'| | `-._`-._ _.-'_.-' | http://redis.io `-._ `-._`-.__.-'_.-' _.-' |`-._`-._ `-.__.-' _.-'_.-'| | `-._`-._ _.-'_.-' | `-._ `-._`-.__.-'_.-' _.-' `-._ `-.__.-' _.-' `-._ _.-' `-.__.-' 1089:X 15 Nov 2020 11:09:41.786 # Sentinel ID is 7a8ec851d30638c286fbfcb916d3b3080070cbdc 1089:X 15 Nov 2020 11:09:41.786 # +monitor master mymaster 127.0.0.1 6380 quorum 2 1089:X 15 Nov 2020 11:09:41.786 * +slave slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6380 1089:X 15 Nov 2020 11:09:41.787 * +slave slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6380 1089:X 15 Nov 2020 11:09:42.235 * +sentinel sentinel 294d622af53ec541a775385baf7cdf99a34f3163 127.0.0.1 26380 @ mymaster 127.0.0.1 6380
从启动日志中,可以看到当前启动的Sentinel也能够发现之前已启动的26380 Sentinel。
-
sentinel_26382.conf
port 26382 pidfile /var/run/redis-sentinel-26382.pid # 监视同一主节点 sentinel monitor mymaster 127.0.0.1 6380 2
2.4 测试
至此,基于Sentinel的Redis HA搭建完成,来测试下功能,将当前redis master节点宕机。
关闭master节点 6380。
^C
699:signal-handler (1605441212) Received SIGINT scheduling shutdown...
699:M 15 Nov 2020 19:53:32.112 # User requested shutdown...
699:M 15 Nov 2020 19:53:32.112 * Saving the final RDB snapshot before exiting.
699:M 15 Nov 2020 19:53:32.119 * DB saved on disk
699:M 15 Nov 2020 19:53:32.119 * Removing the pid file.
699:M 15 Nov 2020 19:53:32.120 # Redis is now ready to exit, bye bye...
master节点19:53:32宕机。
查看Sentinel日志。
1089:X 15 Nov 2020 19:42:32.956 # -tilt #tilt mode exited
1089:X 15 Nov 2020 19:54:02.196 # +sdown master mymaster 127.0.0.1 6380
1089:X 15 Nov 2020 19:54:02.248 # +odown master mymaster 127.0.0.1 6380 #quorum 3/2
1089:X 15 Nov 2020 19:54:02.249 # +new-epoch 1
1089:X 15 Nov 2020 19:54:02.249 # +try-failover master mymaster 127.0.0.1 6380
1089:X 15 Nov 2020 19:54:02.254 # +vote-for-leader 7a8ec851d30638c286fbfcb916d3b3080070cbdc 1
1089:X 15 Nov 2020 19:54:02.257 # 294d622af53ec541a775385baf7cdf99a34f3163 voted for 7a8ec851d30638c286fbfcb916d3b3080070cbdc 1
1089:X 15 Nov 2020 19:54:02.257 # dd6847a566933f15bb8bb04be7f3504f2c00071c voted for 7a8ec851d30638c286fbfcb916d3b3080070cbdc 1
1089:X 15 Nov 2020 19:54:02.313 # +elected-leader master mymaster 127.0.0.1 6380
1089:X 15 Nov 2020 19:54:02.313 # +failover-state-select-slave master mymaster 127.0.0.1 6380
1089:X 15 Nov 2020 19:54:02.377 # +selected-slave slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6380
1089:X 15 Nov 2020 19:54:02.377 * +failover-state-send-slaveof-noone slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6380
1089:X 15 Nov 2020 19:54:02.432 * +failover-state-wait-promotion slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6380
1089:X 15 Nov 2020 19:54:03.062 # +promoted-slave slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6380
1089:X 15 Nov 2020 19:54:03.062 # +failover-state-reconf-slaves master mymaster 127.0.0.1 6380
1089:X 15 Nov 2020 19:54:03.130 * +slave-reconf-sent slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6380
1089:X 15 Nov 2020 19:54:03.402 # -odown master mymaster 127.0.0.1 6380
1089:X 15 Nov 2020 19:54:04.071 * +slave-reconf-inprog slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6380
1089:X 15 Nov 2020 19:54:04.071 * +slave-reconf-done slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6380
1089:X 15 Nov 2020 19:54:04.146 # +failover-end master mymaster 127.0.0.1 6380
1089:X 15 Nov 2020 19:54:04.146 # +switch-master mymaster 127.0.0.1 6380 127.0.0.1 6382
1089:X 15 Nov 2020 19:54:04.147 * +slave slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6382
1089:X 15 Nov 2020 19:54:04.147 * +slave slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6382
1089:X 15 Nov 2020 19:54:34.225 # +sdown slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6382
可以看到:
- 19:54:02.196,Sentinel-26381认为原redis master节点6380主观下线(+sdown master mymaster 127.0.0.1 6380);
- 稍后,经与其他Sentinel确认,原redis master节点6380被标识为客观下线;(1089:X 15 Nov 2020 19:42:32.956 # -tilt #tilt mode exited)
- 准备进行故障转移;(+try-failover master mymaster 127.0.0.1 6380)
- Sentinel-26381被选为Sentinel leader,开始进行故障转移;
- 选举原master节点的从节点6382作为主节点;(+selected-slave slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6380)
- 故障转移结束。
三、使用JedisSentinelPool操作Redis
-
pom依赖
<!-- redis 客户端 --> <dependency> <groupId>redis.clients</groupId> <artifactId>jedis</artifactId> <version>3.1.0</version> </dependency> <!-- 日志组件 --> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-core</artifactId> <version>2.12.0</version> </dependency>
-
RedisUtils
package com.dfx.redisdemos.util; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import redis.clients.jedis.Jedis; import redis.clients.jedis.JedisSentinelPool; import java.util.HashSet; import java.util.Set; /** * @Author dufengxu * @Description * @Date 2020/11/21 14:34 */ public class RedisUtils { private static final Logger LOGGER = LoggerFactory.getLogger(RedisUtils.class); private static JedisSentinelPool sentinelPool; private static final String MASTER_NAME = "mymaster"; static { Set<String> sentinelSet = new HashSet<>(3); sentinelSet.add("127.0.0.1:26380"); sentinelSet.add("127.0.0.1:26381"); sentinelSet.add("127.0.0.1:26382"); // master_name 要与Sentinel监视的master名称一致 // sentinel monitor mymaster 127.0.0.1 6380 2 sentinelPool = new JedisSentinelPool(MASTER_NAME, sentinelSet); } /** * 增加key、val * * @param key * @param val * @return */ public static boolean set(String key, String val) { Jedis jedis = null; try { jedis = sentinelPool.getResource(); jedis.set(key, val); return true; } catch (Exception e) { LOGGER.info("set exception", e); return false; } finally { if (jedis != null) { jedis.close(); } } } /** * 获取key对应的值 * * @param key * @return */ public static String get(String key) { Jedis jedis = null; try { jedis = sentinelPool.getResource(); return jedis.get(key); } catch (Exception e) { LOGGER.info("set exception", e); return null; } finally { if (jedis != null) { jedis.close(); } } } /** * 增加key、val,并设置有效期 * * @param key * @return */ public static boolean set(String key, String val, int expireTimeInSecs) { Jedis jedis = null; try { jedis = sentinelPool.getResource(); jedis.set(key, val); jedis.expire(key, expireTimeInSecs); return true; } catch (Exception e) { LOGGER.info("set exception", e); return false; } finally { if (jedis != null) { jedis.close(); } } } }
【注】在创建JedisSentinelPool时,传入的master name入参,要与Sentinel实际监视的redis master name相一致;
-
使用RedisUtils
public class App { public static void main( String[] args ) { RedisUtils.set("k", "v"); System.out.println(RedisUtils.get("k")); } }