redis技术之旅七

最新推荐文章于 2022-11-12 10:29:16 发布

wow-zyshun

最新推荐文章于 2022-11-12 10:29:16 发布

阅读量620

点赞数

分类专栏： redis技术之旅文章标签： redis

redis技术之旅专栏收录该内容

10 篇文章 0 订阅

订阅专栏

redis技术之旅七 redis sentinel（哨兵）机制的原理和配置
redis sentinel主要功能是在主库（master）出现问题后，实现master存活检测、主从运行情况检测、自动failover、主从切换等高可用。redis的sentinel最小配置是一主一从，实现故障转移高可用。其基本原理就是投票算法、心跳机制。
在哨兵的运行阶段，其会向每个其他的哨兵、master和slave发送消息确认其是否存活，如果在指定的时间内未收到回应，暂时认为对方挂起了（被标记为主观宕机–SDOWN）
当多个哨兵都报告同一个master没有响应了，通过投票算法，系统判断其已死亡（被标记为客观宕机–ODOWN）。在已知的slave节点中，根据实际情况和优先级选出一个slave提升为新的master，其他的slave都指向这个新的master，继续维护主从关系。
redis的sentinel系统可以用来管理多个redis服务器，该系统可以执行以下三个任务：
监控：sentinel会不断的检查你的主服务器和从服务器是否正常运行。
提醒：当被监控的某个redis服务器出现问题，sentinel通过API向管理员或者其他的应用程序发送通知。
自动故障转移：当主服务器不能正常工作时，sentinel会开始一次自动的故障转移操作，它会将与失效主服务器是主从关系的其中一个从库升级为新的主服务器，并且修改其他的的slave，重定向到新的slave。

redis的sentinel是一个分布式系统，可以在一个架构下运行多个sentinel进程，这些进程之间通过流言协议（gossip protocols)来接收关于主服务器是否下线的信息，并使用投票协议（agreement protocols）来决定是否执行自动故障迁移，以及选择哪个从服务器作为新的主服务器。

关于redis sentinel的主观下线和客观下线
redis sentinel关于被监控的redis实例出现不响应的判断，内部有两种不同的概念，主观下线和客观下线
主观下线：当只有单个sentinel实例对redis实例做出无响应的判断，此时进入主观判断，不会触发自动故障转移等操作。
注意，一个服务器必须在 master-down-after-milliseconds 毫秒内，一直返回无效回复才会被 Sentinel 标记为主观下线。
客观下线：多个 Sentinel 实例在对同一个服务器做出 SDOWN 判断，并且通过 SENTINEL is-master-down-by-addr 命令互相交流之后，得出的服务器下线判断。（一个 Sentinel 可以通过向另一个 Sentinel 发送 SENTINEL is-master-down-by-addr 命令来询问对方是否认为给定的服务器已下线。）
从主观下线状态切换到客观下线状态并没有使用严格的法定人数算法（strong quorum algorithm），而是使用了流言协议：如果 Sentinel 在给定的时间范围内，从其他 Sentinel 那里接收到了足够数量的主服务器下线报告，那么 Sentinel 就会将主服务器的状态从主观下线改变为客观下线。如果之后其他 Sentinel 不再报告主服务器已下线，那么客观下线状态就会被移除。
客观下线条件只适用于主服务器：对于任何其他类型的 Redis 实例， Sentinel 在将它们判断为下线前不需要进行协商，所以从服务器或者其他 Sentinel 永远不会达到客观下线条件。
只要一个 Sentinel 发现某个主服务器进入了客观下线状态，这个 Sentinel 就可能会被其他 Sentinel 推选出，并对失效的主服务器执行自动故障迁移操作。

sentinel定时执行的操作
1.每个 Sentinel 以每秒钟一次的频率向它所知的主服务器、从服务器以及其他 Sentinel 实例发送一个 PING 命令。
如果一个实例（instance）距离最后一次有效回复 PING 命令的时间超过 down-after-milliseconds 选项所指定的值，那么这个实例会被 Sentinel 标记为主观下线。一个有效回复可以是： +PONG 、 -LOADING 或者 -MASTERDOWN 。
2.如果一个主服务器被标记为主观下线，那么正在监视这个主服务器的所有 Sentinel 要以每秒一次的频率确认主服务器的确进入了主观下线状态。
3..如果一个主服务器被标记为主观下线，并且有足够数量的 Sentinel （至少要达到配置文件指定的数量）在指定的时间范围内同意这一判断，那么这个主服务器被标记为客观下线。
4.在一般情况下，每个 Sentinel 会以每 10 秒一次的频率向它已知的所有主服务器和从服务器发送 INFO 命令。当一个主服务器被 Sentinel 标记为客观下线时， Sentinel 向下线主服务器的所有从服务器发送 INFO 命令的频率会从 10 秒一次改为每秒一次。
5.当没有足够数量的 Sentinel 同意主服务器已经下线，主服务器的客观下线状态就会被移除。当主服务器重新向 Sentinel 的 PING 命令返回有效回复时，主服务器的主管下线状态就会被移除。

自动发现哨兵（sentinel）和从服务器
sentinel与sentinel之间可以进行信息交换和检测对方的可用性。
1、无需为运行的每个sentinel分别设置其他的sentinel地址，因为sentinel可以通过redis内部的发布\订阅功能来自动的发现正在监视相同主机服务器的其他sentinel，这个功能是通过 sentinel:hello发送消息来实现的。
2、不必列出所有slave的信息，因为sentinel可以通过询问主服务器获取从服务器信息。
每个 Sentinel 会以每两秒一次的频率，通过发布与订阅功能，向被它监视的所有主服务器和从服务器的 sentinel:hello 频道发送一条信息，信息中包含了 Sentinel 的 IP 地址、端口号和运行 ID （runid）。
每个 Sentinel 都订阅了被它监视的所有主服务器和从服务器的 sentinel:hello 频道，查找之前未出现过的 sentinel （looking for unknown sentinels）。当一个 Sentinel 发现一个新的 Sentinel 时，它会将新的 Sentinel 添加到一个列表中，这个列表保存了 Sentinel 已知的，监视同一个主服务器的所有其他 Sentinel 。
Sentinel 发送的信息中还包括完整的主服务器当前配置（configuration）。如果一个 Sentinel 包含的主服务器配置比另一个 Sentinel 发送的配置要旧，那么这个 Sentinel 会立即升级到新配置上。
在将一个新 Sentinel 添加到监视主服务器的列表上面之前， Sentinel 会先检查列表中是否已经包含了和要添加的 Sentinel 拥有相同运行 ID 或者相同地址（包括 IP 地址和端口号）的 Sentinel ，如果是的话， Sentinel 会先移除列表中已有的那些拥有相同运行 ID 或者相同地址的 Sentinel ，然后再添加新 Sentinel

Sentinel 接受的命令：

PING ：返回 PONG 。
SENTINEL masters ：列出所有被监视的主服务器，以及这些主服务器的当前状态。
SENTINEL slaves ：列出给定主服务器的所有从服务器，以及这些从服务器的当前状态。
SENTINEL get-master-addr-by-name ：返回给定名字的主服务器的 IP 地址和端口号。如果这个主服务器正在执行故障转移操作，或者针对这个主服务器的故障转移操作已经完成，那么这个命令返回新的主服务器的 IP 地址和端口号。
SENTINEL reset ：重置所有名字和给定模式 pattern 相匹配的主服务器。 pattern 参数是一个 Glob 风格的模式。重置操作清除主服务器目前的所有状态，包括正在执行中的故障转移，并移除目前已经发现和关联的，主服务器的所有从服务器和 Sentinel 。
SENTINEL failover ：当主服务器失效时，在不询问其他 Sentinel 意见的情况下，强制开始一次自动故障迁移（不过发起故障转移的 Sentinel 会向其他 Sentinel 发送一个新的配置，其他 Sentinel 会根据这个配置进行相应的更新）。

配置前sentinel配置文件学习
上面说了redis sentinel的一些基本实现原理，在配置sentinel之前，先来看看其配置文件的内容（摘自redis 2.8.21）。
Example sentinel.conf
port
The port that this sentinel instance will run on
port 26379

sentinel announce-ip
sentinel announce-port

The above two configuration directives are useful in environments where,
because of NAT, Sentinel is reachable from outside via a non-local address.

When announce-ip is provided, the Sentinel will claim the specified IP address
in HELLO messages used to gossip its presence, instead of auto-detecting the
local address as it usually does.

Similarly when announce-port is provided and is valid and non-zero, Sentinel
will announce the specified TCP port.

The two options don’t need to be used together, if only announce-ip is
provided, the Sentinel will announce the specified IP and the server port
as specified by the “port” option. If only announce-port is provided, the
Sentinel will announce the auto-detected local IP and the specified port.

Example:

sentinel announce-ip 1.2.3.4

dir
Every long running process should have a well-defined working directory.
For Redis Sentinel to chdir to /tmp at startup is the simplest thing
for the process to don’t interfere with administrative tasks such as
unmounting filesystems.
dir /tmp

sentinel monitor

Tells Sentinel to monitor this master, and to consider it in O_DOWN
(Objectively Down) state only if at least sentinels agree.

Note that whatever is the ODOWN quorum, a Sentinel will require to
be elected by the majority of the known Sentinels in order to
start a failover, so no failover can be performed in minority.

Slaves are auto-discovered, so you don’t need to specify slaves in
any way. Sentinel itself will rewrite this configuration file adding
the slaves using additional configuration options.
Also note that the configuration file is rewritten when a
slave is promoted to master.

Note: master name should not include special characters or spaces.
The valid charset is A-z 0-9 and the three characters “.-_”.
sentinel monitor mymaster 127.0.0.1 6379 2

sentinel auth-pass

Set the password to use to authenticate with the master and slaves.
Useful if there is a password set in the Redis instances to monitor.

Note that the master password is also used for slaves, so it is not
possible to set a different password in masters and slaves instances
if you want to be able to monitor these instances with Sentinel.

However you can have Redis instances without the authentication enabled
mixed with Redis instances requiring the authentication (as long as the
password set is the same for all the instances requiring the password) as
the AUTH command will have no effect in Redis instances with authentication
switched off.

Example:

sentinel auth-pass mymaster MySUPER–secret-0123passw0rd

sentinel down-after-milliseconds

Number of milliseconds the master (or any attached slave or sentinel) should
be unreachable (as in, not acceptable reply to PING, continuously, for the
specified period) in order to consider it in S_DOWN state (Subjectively
Down).

Default is 30 seconds.
sentinel down-after-milliseconds mymaster 30000

sentinel parallel-syncs

How many slaves we can reconfigure to point to the new slave simultaneously
during the failover. Use a low number if you use the slaves to serve query
to avoid that all the slaves will be unreachable at about the same
time while performing the synchronization with the master.
sentinel parallel-syncs mymaster 1

sentinel failover-timeout

Specifies the failover timeout in milliseconds. It is used in many ways:

The time needed to re-start a failover after a previous failover was
already tried against the same master by a given Sentinel, is two
times the failover timeout.
The time needed for a slave replicating to a wrong master according
to a Sentinel current configuration, to be forced to replicate
with the right master, is exactly the failover timeout (counting since
the moment a Sentinel detected the misconfiguration).
The time needed to cancel a failover that is already in progress but
did not produced any configuration change (SLAVEOF NO ONE yet not
acknowledged by the promoted slave).
The maximum time a failover in progress waits for all the slaves to be
reconfigured as slaves of the new master. However even after this time
the slaves will be reconfigured by the Sentinels anyway, but not with
the exact parallel-syncs progression as specified.

Default is 3 minutes.
sentinel failover-timeout mymaster 180000

SCRIPTS EXECUTION

sentinel notification-script and sentinel reconfig-script are used in order
to configure scripts that are called to notify the system administrator
or to reconfigure clients after a failover. The scripts are executed
with the following rules for error handling:

If script exits with “1” the execution is retried later (up to a maximum
number of times currently set to 10).

If script exits with “2” (or an higher value) the script execution is
not retried.

If script terminates because it receives a signal the behavior is the same
as exit code 1.

A script has a maximum running time of 60 seconds. After this limit is
reached the script is terminated with a SIGKILL and the execution retried.

NOTIFICATION SCRIPT

sentinel notification-script

Call the specified notification script for any sentinel event that is
generated in the WARNING level (for instance -sdown, -odown, and so forth).
This script should notify the system administrator via email, SMS, or any
other messaging system, that there is something wrong with the monitored
Redis systems.

The script is called with just two arguments: the first is the event type
and the second the event description.

The script must exist and be executable in order for sentinel to start if
this option is provided.

Example:

sentinel notification-script mymaster /var/redis/notify.sh

CLIENTS RECONFIGURATION SCRIPT

sentinel client-reconfig-script

When the master changed because of a failover a script can be called in
order to perform application-specific tasks to notify the clients that the
configuration has changed and the master is at a different address.

The following arguments are passed to the script:

is currently always “failover”
is either “leader” or “observer”

The arguments from-ip, from-port, to-ip, to-port are used to communicate
the old address of the master and the new address of the elected slave
(now a master).

This script should be resistant to multiple invocations.

Example:

sentinel client-reconfig-script mymaster /var/redis/reconfig.sh

部分参数说明：
port ：sentinel实例之间通讯的端口

dir :指定工作目录

sentinel monitor：sentinel需要监控的主库信息 sentinel monitor
其中master-name为自定义master名称，ip为master所在主机的ip地址，redis-port为redis实例的端口号，quorum界定有多少个sentinel实例提交与master通信失败才会判断master为客观宕机（ODOWN），从而发起自动切换。

sentinel auth-pass：如果master开启的密码验证，在这里配置master的密码 sentinel auth-pass

sentinel down-after-milliseconds：master被当前sentinel判断为失效的时间间隔，sentinel与master之间的通信没有响应或者代码错误等超过这个时间限定，sentinel会判断master为客观宕机SDOWNdown-after-milliseconds
sentinel parallel-syncs：当自动切换完成后，同时进行slaveof到新的master并行执行SYNC的slave个数，默认为1，建议线上保留这个数字，在slave执行slaveof的时候，将不会对客户端请求进行响应，对于读写分离业务会有一定的影响
sentinel parallel-syncs

sentinel failover-timeout：制定failover的过期时间，超过此时间没有触发任何的failover操作，当前的sentinel会认为此次的failover擦走哦失败。 sentinel failover-timeout

sentinel notification-script ：当进行failover时，可以指定一个通知脚本用来通知系统管理员，当前集群的情况。脚本被允许执行的最大时间为60秒，超过这个时间，脚本会被kill。 sentinel notification-script
1 -> 稍后重试，最大重试次数为10;
2 -> 执行结束，无需重试

sentinel client-reconfig-script：
failover之后重配置客户端，执行脚本时会传递大量参数，请参考相关文档

参数文件实例
本文使用的redis版本为2.8.21,此版本安装已经自带了sentinel，无需自己在编译安装
新建三个redis实例配置一主两从环境，关于主从的配置，前文已经介绍过这里不做介绍。
端口分别为6379 6479 6579 ，6379为master 、6479 6579为slave
上文已经对sentinel的配置文件讲解过，这里直接开始配置sentinel实例。摘录6579下的sentinel配置文件，除了端口不一样，其他配置的所有sentinel实例的配置文件都一样。
Example sentinel.conf

port
The port that this sentinel instance will run on
port 26579

sentinel announce-ip
sentinel announce-port

The above two configuration directives are useful in environments where,
because of NAT, Sentinel is reachable from outside via a non-local address.

When announce-ip is provided, the Sentinel will claim the specified IP address
in HELLO messages used to gossip its presence, instead of auto-detecting the
local address as it usually does.

Similarly when announce-port is provided and is valid and non-zero, Sentinel
will announce the specified TCP port.

Example:

sentinel announce-ip 1.2.3.4

sentinel monitor

Tells Sentinel to monitor this master, and to consider it in O_DOWN
(Objectively Down) state only if at least sentinels agree.

Note that whatever is the ODOWN quorum, a Sentinel will require to
be elected by the majority of the known Sentinels in order to
start a failover, so no failover can be performed in minority.

Note: master name should not include special characters or spaces.
The valid charset is A-z 0-9 and the three characters “.-_”.
sentinel monitor mymaster 172.21.24.20 6379 2

sentinel auth-pass

Set the password to use to authenticate with the master and slaves.
Useful if there is a password set in the Redis instances to monitor.

Example:

sentinel auth-pass mymaster MySUPER–secret-0123passw0rd

sentinel down-after-milliseconds

Default is 30 seconds.
sentinel down-after-milliseconds mymaster 10000

sentinel parallel-syncs

sentinel failover-timeout

Specifies the failover timeout in milliseconds. It is used in many ways:

The time needed to re-start a failover after a previous failover was
already tried against the same master by a given Sentinel, is two
times the failover timeout.
The time needed for a slave replicating to a wrong master according
to a Sentinel current configuration, to be forced to replicate
with the right master, is exactly the failover timeout (counting since
the moment a Sentinel detected the misconfiguration).
The time needed to cancel a failover that is already in progress but
did not produced any configuration change (SLAVEOF NO ONE yet not
acknowledged by the promoted slave).
The maximum time a failover in progress waits for all the slaves to be
reconfigured as slaves of the new master. However even after this time
the slaves will be reconfigured by the Sentinels anyway, but not with
the exact parallel-syncs progression as specified.

Default is 3 minutes.
sentinel failover-timeout mymaster 180000

SCRIPTS EXECUTION

sentinel notification-script and sentinel reconfig-script are used in order
to configure scripts that are called to notify the system administrator
or to reconfigure clients after a failover. The scripts are executed
with the following rules for error handling:

If script exits with “1” the execution is retried later (up to a maximum
number of times currently set to 10).

If script exits with “2” (or an higher value) the script execution is
not retried.

If script terminates because it receives a signal the behavior is the same
as exit code 1.

A script has a maximum running time of 60 seconds. After this limit is
reached the script is terminated with a SIGKILL and the execution retried.

NOTIFICATION SCRIPT

sentinel notification-script

Call the specified notification script for any sentinel event that is
generated in the WARNING level (for instance -sdown, -odown, and so forth).
This script should notify the system administrator via email, SMS, or any
other messaging system, that there is something wrong with the monitored
Redis systems.

The script is called with just two arguments: the first is the event type
and the second the event description.

The script must exist and be executable in order for sentinel to start if
this option is provided.

Example:

sentinel notification-script mymaster /var/redis/notify.sh

CLIENTS RECONFIGURATION SCRIPT

sentinel client-reconfig-script

When the master changed because of a failover a script can be called in
order to perform application-specific tasks to notify the clients that the
configuration has changed and the master is at a different address.

The following arguments are passed to the script:

is currently always “failover”
is either “leader” or “observer”

The arguments from-ip, from-port, to-ip, to-port are used to communicate
the old address of the master and the new address of the elected slave
(now a master).

This script should be resistant to multiple invocations.

Example:

sentinel client-reconfig-script mymaster /var/redis/reconfig.sh

开始启动sentinel
启动信息如下：
[root@VM-TEMPLATE-M2 /data01/redis/redis6579]
redis-server –include /data01/redis/redis6379/redis6379.conf
[root@VM-TEMPLATE-M2 /data01/redis/redis6579]
redis-sentinel /data01/redis/redis6379/sentinel6379.conf &
[1] 19782
[root@VM-TEMPLATE-M2 /data01/redis/redis6579]
.
_.-__ ''-._ _.- .. ”-. Redis 2.8.21 (00000000/0) 64 bit
.-.-```. ```\/ _.,_ ''-._ ( ' , .-` | `, ) Running in sentinel mode |`-._`-...-` __...-.-._|’_.-'| Port: 26379 |-._ ._ / _.-' | PID: 19782-._ -._-./ .-’ .-’
|-._-._ -.__.-' _.-'_.-'| |-._-._ _.-'_.-' | http://redis.io-._ -._-._.-‘.-’ _.-’
|-._-._ -.__.-' _.-'_.-'| |-._-._ _.-'_.-' |-._ -._-._.-‘.-’ _.-’
-._-._.-’ .-’
-._ _.-'-.__.-’

[19782] 17 Aug 13:38:07.717 Sentinel runid is dae505972c77ff001de81cd618102c16462eff52
[19782] 17 Aug 13:38:07.717 +monitor master mymaster 172.21.24.20 6379 quorum 2

[root@VM-TEMPLATE-M2 /data01/redis/redis6579]
redis-server –include /data01/redis/redis6479/redis6479.conf
[root@VM-TEMPLATE-M2 /data01/redis/redis6579]
ps -ef | grep redis
root 19557 1 0 13:36 ? 00:00:00 redis-server *:6379
root 19782 19364 0 13:38 pts/2 00:00:00 redis-sentinel *:26379
root 19848 1 0 13:38 ? 00:00:00 redis-server *:6479
root 19871 19364 0 13:38 pts/2 00:00:00 grep redis
[root@VM-TEMPLATE-M2 /data01/redis/redis6579]
redis-server –include /data01/redis/redis6479/redis6479.conf
[19782] 17 Aug 13:38:37.793 * +slave slave 172.21.24.20:6479 172.21.24.20 6479 @ mymaster 172.21.24.20 6379
[root@VM-TEMPLATE-M2 /data01/redis/redis6579]
redis-sentinel /data01/redis/redis6479/sentinel6479.conf &
[2] 19951
[root@VM-TEMPLATE-M2 /data01/redis/redis6579]
.
_.-__ ''-._ _.- .. ”-. Redis 2.8.21 (00000000/0) 64 bit
.-.-```. ```\/ _.,_ ''-._ ( ' , .-` | `, ) Running in sentinel mode |`-._`-...-` __...-.-._|’_.-'| Port: 26479 |-._ ._ / _.-' | PID: 19951-._ -._-./ .-’ .-’
|-._-._ -.__.-' _.-'_.-'| |-._-._ _.-'_.-' | http://redis.io-._ -._-._.-‘.-’ _.-’
|-._-._ -.__.-' _.-'_.-'| |-._-._ _.-'_.-' |-._ -._-._.-‘.-’ _.-’
-._-._.-’ .-’
-._ _.-'-.__.-’

[19951] 17 Aug 13:39:02.670 Sentinel runid is da98f745ab50e7956b8f089e77e74e4e8bc1e588
[19951] 17 Aug 13:39:02.670 +monitor master mymaster 172.21.24.20 6379 quorum 2
[19951] 17 Aug 13:39:02.671 * +slave slave 172.21.24.20:6479 172.21.24.20 6479 @ mymaster 172.21.24.20 6379
[19951] 17 Aug 13:39:03.060 * +sentinel sentinel 172.21.24.20:26379 172.21.24.20 26379 @ mymaster 172.21.24.20 6379
[19782] 17 Aug 13:39:04.713 * +sentinel sentinel 172.21.24.20:26479 172.21.24.20 26479 @ mymaster 172.21.24.20 6379

[root@VM-TEMPLATE-M2 /data01/redis/redis6579]
redis-server –include /data01/redis/redis6579/redis6579.conf
[root@VM-TEMPLATE-M2 /data01/redis/redis6579]
ps -ef | grep redis
root 19557 1 0 13:36 ? 00:00:00 redis-server *:6379
root 19782 19364 0 13:38 pts/2 00:00:00 redis-sentinel *:26379
root 19848 1 0 13:38 ? 00:00:00 redis-server *:6479
root 19951 19364 0 13:39 pts/2 00:00:00 redis-sentinel *:26479
root 20040 1 0 13:39 ? 00:00:00 redis-server *:6579
root 20061 19364 0 13:39 pts/2 00:00:00 grep redis

[root@VM-TEMPLATE-M2 /data01/redis/redis6579]
[19782] 17 Aug 13:39:38.001 * +slave slave 172.21.24.20:6579 172.21.24.20 6579 @ mymaster 172.21.24.20 6379
[19951] 17 Aug 13:39:42.777 * +slave slave 172.21.24.20:6579 172.21.24.20 6579 @ mymaster 172.21.24.20 6379
ps -ef | grep redis
root 19557 1 0 13:36 ? 00:00:00 redis-server *:6379
root 19782 19364 0 13:38 pts/2 00:00:00 redis-sentinel *:26379
root 19848 1 0 13:38 ? 00:00:00 redis-server *:6479
root 19951 19364 0 13:39 pts/2 00:00:00 redis-sentinel *:26479
root 20040 1 0 13:39 ? 00:00:00 redis-server *:6579
root 20092 19364 0 13:39 pts/2 00:00:00 grep redis
[root@VM-TEMPLATE-M2 /data01/redis/redis6579]
redis-sentinel /data01/redis/redis6579/sentinel6579.conf &
[3] 20139

[root@VM-TEMPLATE-M2 /data01/redis/redis6579]
.
_.-__ ''-._ _.- .. ”-. Redis 2.8.21 (00000000/0) 64 bit
.-.-```. ```\/ _.,_ ''-._ ( ' , .-` | `, ) Running in sentinel mode |`-._`-...-` __...-.-._|’_.-'| Port: 26579 |-._ ._ / _.-' | PID: 20139-._ -._-./ .-’ .-’
|-._-._ -.__.-' _.-'_.-'| |-._-._ _.-'_.-' | http://redis.io-._ -._-._.-‘.-’ _.-’
|-._-._ -.__.-' _.-'_.-'| |-._-._ _.-'_.-' |-._ -._-._.-‘.-’ _.-’
-._-._.-’ .-’
-._ _.-'-.__.-’

[20139] 17 Aug 13:40:01.747 Sentinel runid is d9b705572ae82c07cb8028daa50037c5c66083a2
[20139] 17 Aug 13:40:01.747 +monitor master mymaster 172.21.24.20 6379 quorum 2
[20139] 17 Aug 13:40:01.748 * +slave slave 172.21.24.20:6479 172.21.24.20 6479 @ mymaster 172.21.24.20 6379
[20139] 17 Aug 13:40:01.749 * +slave slave 172.21.24.20:6579 172.21.24.20 6579 @ mymaster 172.21.24.20 6379
[20139] 17 Aug 13:40:01.879 * +sentinel sentinel 172.21.24.20:26479 172.21.24.20 26479 @ mymaster 172.21.24.20 6379
[20139] 17 Aug 13:40:02.496 * +sentinel sentinel 172.21.24.20:26379 172.21.24.20 26379 @ mymaster 172.21.24.20 6379
[19951] 17 Aug 13:40:03.746 * +sentinel sentinel 172.21.24.20:26579 172.21.24.20 26579 @ mymaster 172.21.24.20 6379