Redis持久化机制

Redis虽然是个内存数据库，但是Redis支持RDB和AOF两种持久化机制，将数据写往磁盘，可以有效地避免因进程退出造成的数据丢失问题，当下次重启时利用之前持久化的文件即可实现数据恢复。

RDB（Redis DataBase）

RDB持久化是把当前进程数据生成快照保存到硬盘的过程。所谓内存快照，就是指内存中的数据在某一个时刻的状态记录。这就像是拍照片。RDB 就是Redis DataBase 的缩写。

用法：save 和 bgsave 命令

区别：save（会导致主线程阻塞） bgsave（会创建子进程来落盘）

什么情况下会自动RDB？

1、客户端执行shutdown命令时，没有开启AOF的时候。

2、使用save相关配置,如“save <seconds> <changes>，下面的配置文件有提到。

3、执行debug reload命令重新启动Redis 时。

4、从节点执行全量复制操作，主节点自动执行bgsave生成RDB文件并发送给从节点。

RDB核心配置

# Save the DB to disk.
#
# save <seconds> <changes>
#
# Redis will save the DB if both the given number of seconds and the given
# number of write operations against the DB occurred.
#
# Snapshotting can be completely disabled with a single empty string argument
# as in following example:
#
# save ""
#
# Unless specified otherwise, by default Redis will save the DB:
#   * After 3600 seconds (an hour) if at least 1 key changed
#   * After 300 seconds (5 minutes) if at least 100 keys changed
#   * After 60 seconds if at least 10000 keys changed
#
# You can set these explicitly by uncommenting the three following lines.
#
# save 秒 次数
save 3600 1
save 300 100
save 60 10000

# 设置RDB的文件路径与名字
# The filename where to dump the DB
dbfilename dump.rdb

# 是否开启LZF压缩RDB
# Compress string objects using LZF when dump .rdb databases?
# By default compression is enabled as it's almost always a win.
# If you want to save some CPU in the saving child set it to 'no' but
# the dataset will likely be bigger if you have compressible values or keys.
rdbcompression yes

RDB执行过程简述

使用bgsave进行RDB时，不会阻塞执行命令的主线程，那这个会借助操作系统提供的写时复制（Copy-On-Write），保证主线程正常读写。

在执行bgsave后，键值对F1所在的内存正在被快照中，如果要修改该键值对，那就把F1 Copy一份改他的副本，改完再把副本写入RDB文件。

RDB优缺点使用场景简述

优点：紧凑的二进制文件，恢复数据时比AOF快，属于快照数据。

缺点：无法保证实时性，不同版本的Redis存在兼容性问题，属于重量级操作（需要fork子进程）

使用场景：用于全量备份，灾难恢复。

AOF(append only file)

以独立日志的方式记录每次写命令，重启时再重新执行AOF文件中的命令达到恢复数据的目的。AOF的主要作用是解决了数据持久化的实时性,目前已经是Redis持久化的主流方式。理解掌握好AOF持久化机制对我们兼顾数据安全性和性能非常有帮助。

用法：下面用配置文件给大家解释

############################## APPEND ONLY MODE ###############################

# By default Redis asynchronously dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).
#
# The Append Only File is an alternative persistence mode that provides
# much better durability. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Please check https://redis.io/topics/persistence for more information.
# 开启AOF
appendonly yes

# The name of the append only file (default: "appendonly.aof")
# AOF路径与文件名
appendfilename "appendonly.aof"

# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".
# Redis支持三种不同的模式:no:不进行fsync，只让操作系统在需要的时候刷新数据。
# 得更快。总是:每次写只追加日志后fsync。缓慢的,安全的。
# Everysec:每秒只执行一次fsync。妥协。
# 默认是“everysec”，因为这通常是速度和数据安全之间的正确折衷。
# 这取决于您是否可以将其放宽为“no”，以便操作系统在需要的时候刷新输出缓冲区，
# 以获得更好的性能(但如果您可以接受一些数据丢失的想法，请考虑默认的持久性模式，即快照)，
# 或者相反，
# 使用“always”，它非常慢，但比everysec更安全一点。如果不确定，请使用“everysec”。

# appendfsync always
appendfsync everysec # 默认的每秒一次把AOF缓冲区的数据刷入磁盘
# appendfsync no

# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync none". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.

no-appendfsync-on-rewrite no

# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.
# 自动重写仅追加文件。当AOF日志大小以指定的百分比增长时，Redis能够自动重写日志文件隐式调用BGREWRITEAOF。
# 它是这样工作的:Redis记住最新重写后AOF文件的大小(如果重新启动后没有发生重写，则使用启动时AOF文件的大小)。此基本大小与当前大小进行比较。
# 如果当前大小大于指定的百分比，则触发重写。此外，您还需要为要重写的AOF文件指定一个最小大小，
# 这对于避免重写AOF文件非常有用，即使达到了百分比增长，但它仍然非常小。指定一个百分比为零，以禁用自动AOF重写功能。

auto-aof-rewrite-percentage 100 #　超过上次重写后大小的一倍就执行
auto-aof-rewrite-min-size 64mb #　AOF文件超过64MB就重写

# An AOF file may be found to be truncated at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can't happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted in the middle
# the server will still exit with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
aof-load-truncated yes

# When rewriting the AOF file, Redis is able to use an RDB preamble in the
# AOF file for faster rewrites and recoveries. When this option is turned
# on the rewritten AOF file is composed of two different stanzas:
#
#   [RDB file][AOF tail]
#
# When loading, Redis recognizes that the AOF file starts with the "REDIS"
# string and loads the prefixed RDB file, then continues loading the AOF
# tail.
# 加载时，Redis识别AOF文件以“Redis”字符串开始，加载前缀RDB文件，然后继续加载AOF尾部。
# 开启AOF＋RDB混合持久化
aof-use-rdb-preamble yes

在配置文件中已经把AOF的核心功能给展示出来了

核心功能简述

１、AOF缓冲区

Redis支持三种不同的模式:no:不进行fsync，只让操作系统在需要的时候刷新数据。得更快。总是:每次写只追加日志后fsync。缓慢的,安全的。
Everysec:每秒只执行一次fsync。妥协。
默认是“everysec”，因为这通常是速度和数据安全之间的正确折衷。
这取决于您是否可以将其放宽为“no”，以便操作系统在需要的时候刷新输出缓冲区，
以获得更好的性能(但如果您可以接受一些数据丢失的想法，请考虑默认的持久性模式，即快照)，
或者相反，使用“always”，它非常慢，但比everysec更安全一点。如果不确定，请使“everysec”。

# appendfsync always # 每次执行指令都立即刷入磁盘
appendfsync everysec # 默认的每秒一次把AOF缓冲区的数据刷入磁盘
# appendfsync no # 让操作系统决定什么时候执行

２、AOF重写

自动重写仅追加文件。当AOF日志大小以指定的百分比增长时，Redis能够自动重写日志文件隐式调用BGREWRITEAOF。它是这样工作的:Redis记住最新重写后AOF文件的大小(如果重新启动后没有发生重写，则使用启动时AOF文件的大小)。此基本大小与当前大小进行比较。
如果当前大小大于指定的百分比，则触发重写。此外，您还需要为要重写的AOF文件指定一个最小大小，这对于避免重写AOF文件非常有用，即使达到了百分比增长，但它仍然非常小。指定一个百分比为零，以禁用自动AOF重写功能。

auto-aof-rewrite-percentage 100 #　超过上次重写后大小的一倍（100%）就执行
auto-aof-rewrite-min-size 64mb #　AOF文件超过64MB就重写

如果在Redis在进行AOF重写时，有写入操作，这个操作也会被写到重写日志的缓冲区。这样，重写日志也不会丢失最新的操作。

３、AOF＋RDB混合持久化

加载时，Redis识别AOF文件以“Redis”字符串开始，加载前缀RDB文件，然后继续加载AOF尾部。
aof-use-rdb-preamble yes # 开启AOF＋RDB混合持久化 

 Redis持久化过程中有没有其他潜在的阻塞风险？

当Redis做RDB或AOF重写时，需要执行fork操作创建子进程,fork是个重量级操作。虽然fork创建的子进程不需要拷贝父进程的物理内存空间，但是会复制父进程的空间内存页表。例如对于10GB的Redis进程，需要复制大约20MB的内存页表，因此fork操作耗时跟进程总内存量息息相关，如果使用虚拟化技术，特别是Xen虚拟机,fork操作会更耗时。

fork耗时问题定位:

对于高流量的Redis实例OPS可达5万以上，如果fork操作耗时在秒级别将拖慢Redis几万条命令执行，对线上应用延迟影响非常明显。正常情况下fork耗时应该是每GB消耗20毫秒左右。

可以在 info stats 统计中查 latest_fork_usec 指标获取最近一次fork操作耗时,单位微秒。

127.0.0.1:6379> info stats
# Stats
total_connections_received:29
total_commands_processed:258631
instantaneous_ops_per_sec:0
total_net_input_bytes:184697234
total_net_output_bytes:2750322469
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:2908
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:80726
evicted_keys:0
keyspace_hits:243595
keyspace_misses:5640
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:466
total_forks:2073
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:2
dump_payload_sanitizations:0
total_reads_processed:262277
total_writes_processed:261594
io_threaded_reads_processed:0
io_threaded_writes_processed:0

Redis主线程、子进程、后台线程是什么?

1、主线程任务处理是用BIO的方式进行的，也就是执行命令的线程；

2、用子进程主要是从数据段的复制(RDB等)去考虑的，当数据被修改时会触发写时复制 子进程会复制父进程被修改的数据页；

3、后台线程主要是处理IO、lazyfree(DEL类型命令删除体积较大的键)等，可以用内存共享的方式来处理数据，不用拷贝数据，所以可以用线程的方式；