Redis 持久化

声明: 本系列博文都是总结于《Redis开发与运维》这本书中相关内容;

Redis 支持RDB 和 AOF 两种类型的持久化机制,

1. RDB:

《1》触发机制:

手动触发:

  • save 命令:阻塞当前Redis执行,直到RDB过程完成为止,对于内存比较大的实例,会造成长时间阻塞; 已废弃
  • bgsave 命令:Redis 进行执行 fork 操作创建子进程, RDB持久化过程由子进程负责完成,完成后子进程自动结束,阻塞只发生在 fork 阶段,bgsave 对save命令做了优化,

自动触发:

  • save m n :表示 m 秒内数据集 存在 n 次 修改时,自动触发 bgsave 操作;
  • save 900 1 : 表示 900 秒内 数据集存在 1 次 修改时,触发bgsave操作;

《2》文件保存位置:

  • RDB 文件 以 ${dbfilename} 保存在 ${dir} 目录下;

dbfilename 和 dir 在配置文件 redis.conf 中自行配置即可或者使用 config set dir {newDir} ;config set dbfilename {newFilename} 运行期进行动态执行;

《3》压缩算法:

RDB 文件默认采用 LZF 算法 对生成的RDB文件做压缩处理,默认开启自动压缩,config set rdbcompression {yes|no} 动态修改;

《4》RDB文件正确性检测:

  • redis-check-rdb dbfilename.rdb

《5》优缺点

优点:

  • RDB是一个紧压缩的二进制文件,代表 Redis 在某个时间点上的数据快照,
  • RDB恢复数据时速度比较快,远超AOF的方式;

缺点:

  • RDB是在某个时间点上fork一个子进程对当前redis内存中数据快照进行备份,所以 没有办法做到实时持久化或者说秒级持久化;
  • 存在老版本Redis无法兼容新版本Redis的rdb文件;
################################ SNAPSHOTTING  ################################
#
# Save the DB on disk:
#
#   save <seconds> <changes>
#
#   Will save the DB if both the given number of seconds and the given
#   number of write operations against the DB occurred.
#
#   In the example below the behaviour will be to save:
#   after 900 sec (15 min) if at least 1 key changed
#   after 300 sec (5 min) if at least 10 keys changed
#   after 60 sec if at least 10000 keys changed
#
#   Note: you can disable saving completely by commenting out all "save" lines.
#
#   It is also possible to remove all the previously configured save
#   points by adding a save directive with a single empty string argument
#   like in the following example:
#
#   save ""
# 自动备份触发机制
save 900 1
save 300 10
save 60 10000

# By default Redis will stop accepting writes if RDB snapshots are enabled
# (at least one save point) and the latest background save failed.
# This will make the user aware (in a hard way) that data is not persisting
# on disk properly, otherwise chances are that no one will notice and some
# disaster will happen.
#
# If the background saving process will start working again Redis will
# automatically allow writes again.
#
# However if you have setup your proper monitoring of the Redis server
# and persistence, you may want to disable this feature so that Redis will
# continue to work as usual even if there are problems with disk,
# permissions, and so forth.
# 当bgsave出错时,是否停止writes
stop-writes-on-bgsave-error yes

# Compress string objects using LZF when dump .rdb databases?
# For default that's set to 'yes' as it's almost always a win.
# If you want to save some CPU in the saving child set it to 'no' but
# the dataset will likely be bigger if you have compressible values or keys.
# rdb文件是否进行压缩
rdbcompression yes

# Since version 5 of RDB a CRC64 checksum is placed at the end of the file.
# This makes the format more resistant to corruption but there is a performance
# hit to pay (around 10%) when saving and loading RDB files, so you can disable it
# for maximum performances.
#
# RDB files created with checksum disabled have a checksum of zero that will
# tell the loading code to skip the check.
# rdb文件checksum
rdbchecksum yes

# The filename where to dump the DB
# rdb文件名字
dbfilename dump.rdb

# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# The Append Only File will also be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
# 保存目录
dir ./

2.AOF

Append Only File 持久化:以独立日志的方式记录每次写命令,重启时,重写执行AOF文件中的命令达到回复数据的目的,主要用于解决数据持久化时的实时性,目前已经是 Redis 持久化的主流方式;

开关配置:

appendonly yes : 默认不开启;

文件名称:

appendfilename : 默认文件名:appendonly.aof

保存路径:

dir : 默认当前目录 ./

《1》建议工作流程:

  1. Redis 所有的命令会追加到 aof_buf (缓冲区)中; 之所以先追加到缓冲区是因为,Redis是单进程的,如果每次都追加到硬盘,那么性能完全取决于当前硬盘的负载。
  2. AOF 缓冲区根据对应的策略向硬盘做同步操作;
  3. 随着 AOF 文件越来越大,需要定时对 AOF 文件进行重写,到达压缩目的。
  4. 当 Redis服务器重启时,可以加载AOF文件进行数据恢复;

《2》文件同步策略

  • always : 命令写入到 aof_buf 后调用系统 fsync 操作同步到aop文件,fsync 完成后线程返回;
  • everysec : 命令写入 aof_buf后调用系统write操作,write操作完成线程返回。fsync 同步文件操作由专门的线程 每秒调用一次;
  • no : 命令写入 aof_buf后调用系统write操作,fsync 同步文件操作由系统负责;数据安全性较差;

《3》aop文件重写触发机制

手动触发:

  • 直接调用命令: bgrewriteaof

自动触发:

根据 auto-aof-rewrite-min-size 和 auto-aof-rewrite-percentage 参数来自动触发;

  • auto-aof-rewrite-min-size : 表示运行 AOF 重写时,文件最小最小体积,默认 64mb;
  • auto-aof-rewrite-percentage : 当前 AOF 文件空间(aof_current_size)和上次重写后AOF文件空间(aof_base_size)的比值;

自动触发时机:aof_current_size > auto-auto-rewrite-min-size && (aof_current_size - aof_base_size)/ aof_base_size >= auto_aof_rewrite_percentage;

############################## APPEND ONLY MODE ###############################

# By default Redis asynchronously dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).
#
# The Append Only File is an alternative persistence mode that provides
# much better durability. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Please check http://redis.io/topics/persistence for more information.
# 开启aof持久化
appendonly yes

# The name of the append only file (default: "appendonly.aof")
# aof 持久化文件名称
appendfilename "appendonly.aof"

# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".
#
# aof的输出缓冲区数据刷入磁盘策略;
# appendfsync always
appendfsync everysec
# appendfsync no

# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call(Redis可能阻塞很长时间当调用fsync时).
# Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate(缓解) this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync none". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.
# 如果你存在延迟问题,可以将其设置为yes, 否则让其保持为no,这样更安全
no-appendfsync-on-rewrite no

# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.
# 重写自动触发的两个关键参数
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

# An AOF file may be found to be truncated(缩短) at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can't happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted in the middle
# the server will still exit with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
# 比如当aof文件保存时,突然遇到断电,最后一行没保存全时候,是继续还是退出;
aof-load-truncated yes

# When rewriting the AOF file, Redis is able to use an RDB preamble(序文) in the
# AOF file for faster rewrites and recoveries. When this option is turned
# on the rewritten AOF file is composed of two different stanzas:
#
#   [RDB file][AOF tail]
#
# When loading Redis recognizes that the AOF file starts with the "REDIS"
# string and loads the prefixed RDB file, and continues loading the AOF
# tail.
# 
# 对aof文件重写或者回复时候,可以使用rdb序文,快,默认yes就好
aof-use-rdb-preamble yes

3.AOF RDB

《1》rdb 和 aof 同时存在时,先使用 aof 因为其安全和数据完整性交高;

《2》文件校验:

对于错误的AOF文件,可以使用 redis-check-aof --fix 进行人工修复;然后使用diff -u 对比数据的差异,进行人工修复;

 

 

 

 

 

 

 

 

 

 

 

 

  • 1
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值