Redis学习--------主从复制

1. redis集群的搭建

1.1 方式一

1. 首先启动所有redis节点。
2. 挑选一台节点为主节点。
3. 在所有从节点中执行replicaof 主节点的ip  主节点的端口，在redis旧版本使用slaveof 主节点ip  主节点端口
4. 通过replicaof no one，取消作为从节点。

1.2 方式二

1. 在所有节点中挑选一个作为主节点，然后启动
2. 在启动所有从节点时，redis-server  配置文件路径   --replicaof 主节点ip  主节点端口

1.3 方式三

在从节点配置文件中

 replicaof <masterip> <masterport>

设置主节点信息

2. 主从复制相关配置

设置跟随的主节点：

replica-serve-stale-data yes

在从节点从主节点获取到RDB文件时，需要时间进行恢复数据，在此期间，是否允许从节点仍然使用旧数据响应客户端请求。

# When a replica loses its connection with the master, or when the replication
# is still in progress, the replica can act in two different ways:
#
# 1) if replica-serve-stale-data is set to 'yes' (the default) the replica will
#    still reply to client requests, possibly with out of date data, or the
#    data set may just be empty if this is the first synchronization.
#
# 2) If replica-serve-stale-data is set to 'no' the replica will reply with
#    an error "SYNC with master in progress" to all commands except:
#    INFO, REPLICAOF, AUTH, PING, SHUTDOWN, REPLCONF, ROLE, CONFIG, SUBSCRIBE,
#    UNSUBSCRIBE, PSUBSCRIBE, PUNSUBSCRIBE, PUBLISH, PUBSUB, COMMAND, POST,
#    HOST and LATENCY.
#
replica-serve-stale-data yes

所有的从节点默认对外服务为只读，不能进行修改操作。可以通过此设置进行修改

# You can configure a replica instance to accept writes or not. Writing against
# a replica instance may be useful to store some ephemeral data (because data
# written on a replica will be easily deleted after resync with the master) but
# may also cause problems if clients are writing to it because of a
# misconfiguration.
#
# Since Redis 2.6 by default replicas are read-only.
#
# Note: read only replicas are not designed to be exposed to untrusted clients
# on the internet. It's just a protection layer against misuse of the instance.
# Still a read only replica exports by default all the administrative commands
# such as CONFIG, DEBUG, and so forth. To a limited extent you can improve
# security of read only replicas using 'rename-command' to shadow all the
# administrative / dangerous commands.
replica-read-only yes

 在默认情况下，主节点接受到从节点的全量同步请求时，主节点通过磁盘I/O，执行RDB方式持久化到磁盘文件，然后通过网络IO传输给从节点。当磁盘IO较慢时，可以设置不过磁盘，直接通过网络IO传输。

# Replication SYNC strategy: disk or socket.
#
# New replicas and reconnecting replicas that are not able to continue the
# replication process just receiving differences, need to do what is called a
# "full synchronization". An RDB file is transmitted from the master to the
# replicas.
#s
# The transmission can happen in two different ways:
#
# 1) Disk-backed: The Redis master creates a new process that writes the RDB
#                 file on disk. Later the file is transferred by the parent
#                 process to the replicas incrementally.
# 2) Diskless: The Redis master creates a new process that directly writes the
#              RDB file to replica sockets, without touching the disk at all.
#
# With disk-backed replication, while the RDB file is generated, more replicas
# can be queued and served with the RDB file as soon as the current child
# producing the RDB file finishes its work. With diskless replication instead
# once the transfer starts, new replicas arriving will be queued and a new
# transfer will start when the current one terminates.
#
# When diskless replication is used, the master waits a configurable amount of
# time (in seconds) before starting the transfer in the hope that multiple
# replicas will arrive and the transfer can be parallelized.
#
# With slow disks and fast (large bandwidth) networks, diskless replication
# works better.
repl-diskless-sync no

在主节点中维护了一个队列，在开始生成RDB后，会把接受到所有操作，放入到队列中。这个队列大小可设置。

# Set the replication backlog size. The backlog is a buffer that accumulates
# replica data when replicas are disconnected for some time, so that when a
# replica wants to reconnect again, often a full resync is not needed, but a
# partial resync is enough, just passing the portion of data the replica
# missed while disconnected.
#
# The bigger the replication backlog, the longer the replica can endure the
# disconnect and later be able to perform a partial resynchronization.
#
# The backlog is only allocated if there is at least one replica connected.
#
# repl-backlog-size 1mb

3. 主从复制过程

1. 通过replicaof方式从节点和主节点建立连接。
2. 如果从节点第一次进行主节点的数据同步，会进行数据的全量同步。
3. 主节点会fork一个子进程进行RDB方式数据备份。这个期间内所有的操作存入主节点维护的一个队列中，然后把复制流发送给从节点进行同步，replication ID不变并且修改偏移量。
4. 如果repl-diskless-sync为no会把rdb文件存入磁盘，然后通过网络Io传输给从节点。如果为yes，不会进行磁盘存储，直接经过网络IO传输。
5. 从节点接受到RDB文件后会进行同步数据。在同步数据时，从节点会拿这旧数据进行相应客户端请求。当数据同步完成后会把旧数据全部删除。
6. 从节点会根据replication ID和偏移量向主节点的队列中请求指令数据，然后进行增量数据的同步。
7. 如果从节点出现故障，再次启动时，会首先根据replication ID和偏移量区主节点中尝试进行增量数据同步，因为队列容量有限，如果队列中不存在这个偏移量，会进行全量数据的同步。

全量数据同步

主节点：

 4. 总结

4.1 全量同步和增量同步的时机

发生全量同步的情况：

1. 当从节点的replication ID 和 主节点的replicationID不同时，会进行全量数据同步

2. 当从节点的replicationID 和主节点的replicationID相同时，但是从节点的同步的offset在循环队列里面已经不存在了，此时会进行全量同步

 发生增量同步的情况：

1.从节点的replicationID和主节点的replicationID相同，并且从节点的offset偏移量在循环队列中存在，此时会进行增量数据同步

4.2 如何优化主从复制

1. 当从节点服务宕机或重启时应该尽快进行修复启动，减少主节点的循环队列覆盖从节点的offset，导致全量数据同步

2. 设置合适的循环队列大小（repl-backlog-size），减少offset覆盖。

3. 开启repl-diskless-sync，不存储RDB文件，生成的数据直接网络传输到从节点，前提网络io较好，否则可能会导致网络堵塞。