redis 流水笔记四 持久化

1、 RDB 与 AOF

  什么是RDB （redis database)

   RDB 是一个紧凑型单文件系，时间点表示的redis data。

         单个文件便于从灾难快速恢复

         启动一个子进程处理备份，不影响主进程写入。

         允许快速恢复从大的数据集中，相对于aof。

• RDB is a very compact single-file point-in-time representation of your Redis data. RDB files are perfect for backups. For instance you may want to archive your RDB files every hour for the latest 24 hours, and to save an RDB snapshot every day for 30 days. This allows you to easily restore different versions of the data set in case of disasters.
• RDB is very good for disaster recovery, being a single compact file that can be transferred to far data centers, or onto Amazon S3 (possibly encrypted).
• RDB maximizes Redis performances since the only work the Redis parent process needs to do in order to persist is forking a child that will do all the rest. The parent instance will never perform disk I/O or alike.
• RDB allows faster restarts with big datasets compared to AOF.

   有哪些缺点

    RDB 是时间点触发的操作，可能会丢失最后一次时间的数据。

    RDB 会启动一个fork 一个子进程处理，但是如果dataset数据大，会影响client的数据写入，内存占用将会翻倍。

• RDB is NOT good if you need to minimize the chance of data loss in case Redis stops working (for example after a power outage). You can configure different save points where an RDB is produced (for instance after at least five minutes and 100 writes against the data set, but you can have multiple save points). However you'll usually create an RDB snapshot every five minutes or more, so in case of Redis stopping working without a correct shutdown for any reason you should be prepared to lose the latest minutes of data.
• RDB needs to fork() often in order to persist on disk using a child process. Fork() can be time consuming if the dataset is big, and may result in Redis to stop serving clients for some millisecond or even for one second if the dataset is very big and the CPU performance not great. AOF also needs to fork() but you can tune how often you want to rewrite your logs without any trade-off on durability.

  如何触发 RDB

     1、redis.conf  配置文件配置。规则

     2、save 会阻塞写入，有先backup ，save 命令执行后，会立即执行backup操作。

     3、bgsave 不会影响写入。

     4、 flushall 命令也会触发，dump.rdb 文件生成，不过是生成的空，毫无意义，慎用。

 如何恢复

     1、重新启动配置文件 dumpfilename  dump.rdb  配置为备份的文件即可。

 rdb文件损坏怎么办？

     1、使用redis-check-rdb  --fix  dump.rdb 

 压缩

     1、 使用的是LZF 算法进行压缩CRC64，压缩会损耗cpu 10% 的性能。

  什么是AOF  （append only file）

      aof  文件追加的方式写数据，数据出现错误可以使用 redis-check-aof --fix  appendOnlyfile.aof 进行修复。

      可以在配置文件中配置重写。

• Using AOF Redis is much more durable: you can have different fsync policies: no fsync at all, fsync every second, fsync at every query. With the default policy of fsync every second write performances are still great (fsync is performed using a background thread and the main thread will try hard to perform writes when no fsync is in progress.) but you can only lose one second worth of writes.
• The AOF log is an append only log, so there are no seeks, nor corruption problems if there is a power outage. Even if the log ends with an half-written command for some reason (disk full or other reasons) the redis-check-aof tool is able to fix it easily.
• Redis is able to automatically rewrite the AOF in background when it gets too big. The rewrite is completely safe as while Redis continues appending to the old file, a completely new one is produced with the minimal set of operations needed to create the current data set, and once this second file is ready Redis switches the two and starts appending to the new one.
• AOF contains a log of all the operations one after the other in an easy to understand and parse format. You can even easily export an AOF file. For instance even if you flushed everything for an error using a FLUSHALLcommand, if no rewrite of the log was performed in the meantime you can still save your data set just stopping the server, removing the latest command, and restarting Redis again.

  2、缺点   

       aof 文件时相对于rdb 文件比较大的。

• AOF files are usually bigger than the equivalent RDB files for the same dataset.
• AOF can be slower than RDB depending on the exact fsync policy. In general with fsync set to every secondperformance is still very high, and with fsync disabled it should be exactly as fast as RDB even under high load. Still RDB is able to provide more guarantees about the maximum latency even in the case of an huge write load.
• In the past we experienced rare bugs in specific commands (for instance there was one involving blocking commands like BRPOPLPUSH) causing the AOF produced to not reproduce exactly the same dataset on reloading. These bugs are rare and we have tests in the test suite creating random complex datasets automatically and reloading them to check everything is fine. However, these kind of bugs are almost impossible with RDB persistence. To make this point more clear: the Redis AOF works by incrementally updating an existing state, like MySQL or MongoDB does, while the RDB snapshotting creates everything from scratch again and again, that is conceptually more robust. However - 1) It should be noted that every time the AOF is rewritten by Redis it is recreated from scratch starting from the actual data contained in the data set, making resistance to bugs stronger compared to an always appending AOF file (or one rewritten reading the old AOF instead of reading the data in memory). 2) We have never had a single report from users about an AOF corruption that was detected in the real world。

  3、 aof 重写策略

    appendfsync    always   每个redis命令都要同步到写入硬盘，严重影响redis的速度

                              everysec 每秒执行一次同步，显示将多个写命令同步到硬盘，可能会丢失1秒的数据

                              no   让操作系统来决定何时同步。

    auto-aof-rewrite-percentage  100%

    auto-aof-rewrite-min-size     64M

    重写策略工作机制：

        Log rewriting uses the same copy-on-write trick already in use for snapshotting. This is how it works:

• Redis forks, so now we have a child and a parent process.

• The child starts writing the new AOF in a temporary file.

• The parent accumulates all the new changes in an in-memory buffer (but at the same time it writes the new changes in the old append-only file, so if the rewriting fails, we are safe).

• When the child is done rewriting the file, the parent gets a signal, and appends the in-memory buffer at the end of the file generated by the child.

• Profit! Now Redis atomically renames the old file into the new one, and starts appending new data into the new file.