Redis---B站学习---redis持久化之AOF

Redis—B站学习—redis持久化之AOF

1.什么是AOF

1. 以日志的形式来记录每个写操作，将Redis执行过的所有写指令记录下来(读操作不记录)，只许追加文件但不可以改写文件，redis启动之初会读取该文件重新构建数据。
2. 换言之，redis重启的话就根据日志文件的内容将写指令从前到后执行一次以完成数据的恢复工作

2.保存文件形式

1. Aof保存的是appendonly.aof文件

3.配置位置(在配置文件中)

1. 配置文件内容

   1. appendonly：配置aof持久化开关
   2. appendfilename：aof持久化文件的默认名字
   3. appendfsync
      1. always：同步持久化 每次发生数据变更会被立即记录到磁盘 性能较差但数据完整性比较好
      2. everysec：出厂默认推荐，异步操作，每秒记录 如果一秒内宕机，有数据丢失
      3. no
   4. no-appendfsync-on-rewrite：重写时是否可以运用Appendfsync，用默认no即可，保证数据安全性。
   5. auto-aof-rewrite-min-size：设置重写的基准值
   6. auto-aof-rewrite-percentage：设置重写的基准值

   ############################## APPEND ONLY MODE ###############################
   
   # By default Redis asynchronously dumps the dataset on disk. This mode is
   # good enough in many applications, but an issue with the Redis process or
   # a power outage may result into a few minutes of writes lost (depending on
   # the configured save points).
   #
   # The Append Only File is an alternative persistence mode that provides
   # much better durability. For instance using the default data fsync policy
   # (see later in the config file) Redis can lose just one second of writes in a
   # dramatic event like a server power outage, or a single write if something
   # wrong with the Redis process itself happens, but the operating system is
   # still running correctly.
   #
   # AOF and RDB persistence can be enabled at the same time without problems.
   # If the AOF is enabled on startup Redis will load the AOF, that is the file
   # with the better durability guarantees.
   #
   # Please check http://redis.io/topics/persistence for more information.
   
   //配置aof持久化的开关，yes是开启，no是关闭
   appendonly no
   
   //默认名字
   # The name of the append only file (default: "appendonly.aof")
   
   appendfilename "appendonly.aof"
   
   //默认的追写策略
   # The fsync() call tells the Operating System to actually write data on disk
   # instead of waiting for more data in the output buffer. Some OS will really flush
   # data on disk, some other OS will just try to do it ASAP.
   #
   # Redis supports three different modes:
   #
   # no: don not fsync, just let the OS flush the data when it wants. Faster.
   # always: fsync after every write to the append only log. Slow, Safest.
   # everysec: fsync only one time every second. Compromise.
   #
   # The default is "everysec", as that is usually the right compromise between
   # speed and data safety. It is up to you to understand if you can relax this to
   # "no" that will let the operating system flush the output buffer when
   # it wants, for better performances (but if you can live with the idea of
   # some data loss consider the default persistence mode that is snapshotting),
   # or on the contrary, use "always" that is very slow but a bit safer than
   # everysec.
   #
   # More details please check the following article:
   # http://antirez.com/post/redis-persistence-demystified.html
   #
   # If unsure, use "everysec".
   
   //默认的追写策略，出场默认everysec
   # appendfsync always
   appendfsync everysec
   # appendfsync no
   
   # When the AOF fsync policy is set to always or everysec, and a background
   # saving process (a background save or AOF log background rewriting) is
   # performing a lot of I/O against the disk, in some Linux configurations
   # Redis may block too long on the fsync() call. Note that there is no fix for
   # this currently, as even performing fsync in a different thread will block
   # our synchronous write(2) call.
   #
   # In order to mitigate this problem it is possible to use the following option
   # that will prevent fsync() from being called in the main process while a
   # BGSAVE or BGREWRITEAOF is in progress.
   #
   # This means that while another child is saving, the durability of Redis is
   # the same as "appendfsync none". In practical terms, this means that it is
   # possible to lose up to 30 seconds of log in the worst scenario (with the
   # default Linux settings).
   #
   # If you have latency problems turn this to "yes". Otherwise leave it as
   # "no" that is the safest pick from the point of view of durability.
   
   //重写时是否可以运用Appendfsync，用默认no即可，保证数据安全性
   no-appendfsync-on-rewrite no
   
   //触发Rewrite机制
   # Automatic rewrite of the append only file.
   # Redis is able to automatically rewrite the log file implicitly calling
   # BGREWRITEAOF when the AOF log size grows by the specified percentage.
   #
   # This is how it works: Redis remembers the size of the AOF file after the
   # latest rewrite (if no rewrite has happened since the restart, the size of
   # the AOF at startup is used).
   #
   # This base size is compared to the current size. If the current size is
   # bigger than the specified percentage, the rewrite is triggered. Also
   # you need to specify a minimal size for the AOF file to be rewritten, this
   # is useful to avoid rewriting the AOF file even if the percentage increase
   # is reached but it is still pretty small.
   #
   # Specify a percentage of zero in order to disable the automatic AOF
   # rewrite feature.
   
   //Redis会记录上次重写时的AOF大小，默认配置是当AOF文件大小是上次rewrite后大小的一倍且文件大于64M时触发
   auto-aof-rewrite-percentage 100
   auto-aof-rewrite-min-size 64mb
   
   # An AOF file may be found to be truncated at the end during the Redis
   # startup process, when the AOF data gets loaded back into memory.
   # This may happen when the system where Redis is running
   # crashes, especially when an ext4 filesystem is mounted without the
   # data=ordered option (however this can't happen when Redis itself
   # crashes or aborts but the operating system still works correctly).
   #
   # Redis can either exit with an error when this happens, or load as much
   # data as possible (the default now) and start if the AOF file is found
   # to be truncated at the end. The following option controls this behavior.
   #
   # If aof-load-truncated is set to yes, a truncated AOF file is loaded and
   # the Redis server starts emitting a log to inform the user of the event.
   # Otherwise if the option is set to no, the server aborts with an error
   # and refuses to start. When the option is set to no, the user requires
   # to fix the AOF file using the "redis-check-aof" utility before to restart
   # the server.
   #
   # Note that if the AOF file will be found to be corrupted in the middle
   # the server will still exit with an error. This option only applies when
   # Redis will try to read more data from the AOF file but not enough bytes
   # will be found.
   aof-load-truncated yes
   
   # When rewriting the AOF file, Redis is able to use an RDB preamble in the
   # AOF file for faster rewrites and recoveries. When this option is turned
   # on the rewritten AOF file is composed of two different stanzas:
   #
   #   [RDB file][AOF tail]
   #
   # When loading Redis recognizes that the AOF file starts with the "REDIS"
   # string and loads the prefixed RDB file, and continues loading the AOF
   # tail.
   aof-use-rdb-preamble yes
   

4.AOF启动/修复/恢复

1. 正常恢复
   1. 启动：设置Yes，修改默认的appendonly no，改为yes
   2. 将有数据的aof文件复制一份保存到对应目录(config get dir)
   3. 恢复：重启redis然后重新加载
2. 异常恢复
   1. 启动：设置Yes，修改默认的appendonly no，改为yes，备份被写坏的AOF文件
   2. 修复：redis-check-aof --fix 文件名，进行修复
   3. 恢复：重启redis然后重新加载

5.Rewrite重写

1. 是什么：
   1. AOF采用文件追加方式，文件会越来越大为避免出现此种情况，新增了重写机制,
   2. 当AOF文件的大小超过所设定的阈值时，Redis就会启动AOF文件的内容压缩，只保留可以恢复数据的最小指令集.
   3. 可以使用命令BGREWRITEAOF
2. 重写原理
   1. AOF文件持续增长而过大时，会fork出一条新进程来将文件重写(也是先写临时文件最后再rename)，遍历新进程的内存中数据，每条记录有一条的Set语句。
   2. 重写aof文件的操作，并没有读取旧的aof文件，而是将整个内存中的数据库内容用命令的方式重写了一个新的aof文件，这点和快照有点类似
3. 触发机制
   Redis会记录上次重写时的AOF大小，默认配置是当AOF文件大小是上次rewrite后大小的一倍且文件大于64M时触发

6.优势

1. 每修改同步：appendfsync always 同步持久化 每次发生数据变更会被立即记录到磁盘 性能较差但数据完整性比较好
2. 每秒同步：appendfsync everysec 异步操作，每秒记录 如果一秒内宕机，有数据丢失
3. 不同步：appendfsync no 从不同步

7.劣势

1. 相同数据集的数据而言aof文件要远大于rdb文件，恢复速度慢于rdb
2. aof运行效率要慢于rdb,每秒同步策略效率较好，不同步效率和rdb相同

8.小总结

9.总结RDB和AOF

1. 官网建议
   
2. RDB持久化方式能够在指定的时间间隔能对你的数据进行快照存储
3. AOF持久化方式记录每次对服务器写的操作,当服务器重启的时候会重新执行这些命令来恢复原始的数据,AOF命令以redis协议追加保存每次写的操作到文件末尾.Redis还能对AOF文件进行后台重写,使得AOF文件的体积不至于过大
4. 只做缓存：如果你只希望你的数据在服务器运行的时候存在,你也可以不使用任何持久化方式.
5. 同时开启两种持久化方式(先加载aof)
   1. 在这种情况下,当redis重启的时候会优先载入AOF文件来恢复原始的数据,
      因为在通常情况下AOF文件保存的数据集要比RDB文件保存的数据集要完整.
   2. RDB的数据不实时，同时使用两者时服务器重启也只会找AOF文件。那要不要只使用AOF呢？ 作者建议不要，因为RDB更适合用于备份数据库(AOF在不断变化不好备份)，快速重启，而且不会有AOF可能潜在的bug，留着作为一个万一的手段。

10.性能建议

1. 因为RDB文件只用作后备用途，建议只在Slave上持久化RDB文件，而且只要15分钟备份一次就够了，只保留save 900 1这条规则。
2. 如果Enalbe AOF，好处是在最恶劣情况下也只会丢失不超过两秒数据，启动脚本较简单只load自己的AOF文件就可以了。代价一是带来了持续的IO，二是AOF rewrite的最后将rewrite过程中产生的新数据写到新文件造成的阻塞几乎是不可避免的。只要硬盘许可，应该尽量减少AOF rewrite的频率，AOF重写的基础大小默认值64M太小了，可以设到5G以上。默认超过原大小100%大小时重写可以改到适当的数值。
3. 如果不Enable AOF ，仅靠Master-Slave Replication 实现高可用性也可以。能省掉一大笔IO也减少了rewrite时带来的系统波动。代价是如果Master/Slave同时倒掉，会丢失十几分钟的数据，启动脚本也要比较两个Master/Slave中的RDB文件，载入较新的那个。新浪微博就选用了这种架构