maxmemory
directive that is used in order to limit the memory usage to a fixed amount, and it also covers in depth the LRU algorithm used by Redis, that is actually an approximation of the exact LRU.
Maxmemory configuration directive 最大内存配置指令
The
maxmemory
configuration directive is used in order to configure Redis to use a specified amount of memory for the data set. It is possible to set the configuration directive using the
redis.conf
file, or later using the
CONFIG SET
command at runtime.
maxmemory
指令是用作配置指定Redis使用内存的最大值。可以在redis.conf文件配置,运行过程中还可以使用CONFIG SET 命令设置。For example in order to configure a memory limit of 100 megabytes, the following directive can be used inside the redis.conf
file.
maxmemory 100mb
Setting
maxmemory
to zero results into no memory limits. This is the default behavior for 64 bit systems, while 32 bit systems use an implicit memory limit of 3GB.
maxmemory
为0 的结果是没有内存限制。这是64位系统的缺省做法,32位系统是限制3G。
Eviction policies 逐出政策
The exact behavior Redis follows when the
maxmemory
limit is reached is configured using the
maxmemory-policy
configuration directive.
- noeviction: return errors when the memory limit was reached and the client is trying to execute commands that could result in more memory to be used (most write commands, but DEL and a few more exceptions).
- noeviction:当达到最大内存使用限制并且客户端尝试执行使用更多内存的命令时,将返回一个错误。
- allkeys-lru: evict keys trying to remove the less recently used (LRU) keys first, in order to make space for the new data added.
- allkeys-lru:为了新添加的数据使用内存,将逐出最近最少使用的key。
- volatile-lru: evict keys trying to remove the less recently used (LRU) keys first, but only among keys that have anexpire set, in order to make space for the new data added.
- volatile-lru: 为了新添加的数据使用内存,将逐出有过期时间设置并且最近最少使用的key。
- allkeys-random: evict random keys in order to make space for the new data added.
- allkeys-random:: 为了新添加的数据使用内存,将随机逐出key。
- volatile-random: evict random keys in order to make space for the new data added, but only evict keys with anexpire set.
- volatile-random: 为了新添加的数据使用内存,将随机逐出具有过期时间设置的key。
- volatile-ttl: In order to make space for the new data, evict only keys with an expire set, and try to evict keys with a shorter time to live (TTL) first.
- 为了新添加的数据使用内存,将逐出有过期时间限制,并且存活时间(TTL)最短的key。
- Use the allkeys-lru policy when you expect a power-law distribution in the popularity of your requests, that is, you expect that a subset of elements will be accessed far more often than the rest. This is a good pick if you are unsure.
- Use the allkeys-random if you have a cyclic access where all the keys are scanned continuously, or when you expect the distribution to be uniform (all elements likely accessed with the same probability).
- Use the volatile-ttl if you want to be able to provide hints to Redis about what are good candidate for expiration by using different TTL values when you create your cache objects.
How the eviction process works 逐出是这样工作的
It is important to understand that the eviction process works like this:
理解逐出工作方式,就像这样:
- A client runs a new command, resulting in more data added.
- Redis checks the memory usage, and if it is greater than the
maxmemory
limit , it evicts keys according to the policy.
maxmemory
限制,就根据政策逐出key。
- A new command is executed, and so forth.
So we continuously cross the boundaries of the memory limit, by going over it, and then by evicting keys to return back under the limits.
因此我们持续添加数据达到内存限制,通过上面的步骤,逐出一些key使得内存回到限制之下。
If a command results in a lot of memory being used (like a big set intersection stored into a new key) for some time the memory limit can be surpassed by a noticeable amount.
如果命令的结果是导致很大的内存被使用(比如一个新的key是一个大的set集合),很多时候达到内存限制是显而易见的。
Approximated LRU algorithm 近似的LRU算法
Redis LRU algorithm is not an exact implementation. This means that Redis is not able to pick the
best candidate
for eviction, that is, the access that was accessed the most in the past. Instead it will try to run an approximation of the LRU algorithm, by sampling a small number of keys, and evicting the one that is the best (with the oldest access time) among the sampled keys.
maxmemory-samples 5
The reason why Redis does not use a true LRU implementation is because it costs more memory. However the approximation is virtually equivalent for the application using Redis. The following is a graphical comparison of how the LRU approximation used by Redis compares with true LRU.
You can see three kind of dots in the graphs, forming three distinct bands.
在上图中你可以看见3个明显的区别:
- The light gray band are objects that were evicted.
- The gray band are objects that were not evicted.
- The green band are objects that were added.
As you can see Redis 3.0 does a better job with 5 samples compared to Redis 2.8, however most objects that are among the latest accessed are still retained by Redis 2.8. Using a sample size of 10 in Redis 3.0 the approximation is very close to the theoretical performance of Redis 3.0.
如你所见,3.0的工作比2.8更好,然而在2.8版本中,大多数最新访问对象的仍然保留。在3.0使用样品为10 时,性能非常接近理论上的LRU算法。
Note that LRU is just a model to predict how likely a given key will be accessed in the future. Moreover, if your data access pattern closely resembles the power law, most of the accesses will be in the set of keys that the LRU approximated algorithm will be able to handle well.
注意:LRU仅仅是一个预测模式,给出的key很可能在未来被访问。此外,如果你的数据访问模式类似于幂律(线性的),大多数key都可能被访问那么这个LRU算法的处理就是非常好的。
In simulations we found that using a power law access pattern, the difference between true LRU and Redis approximation were minimal or non-existent.
在实战中 ,我们发现使用幂律(线性的)的访问模式,在真正的LRU算法和Redis的LRU算法之间差异很小或者不存在差异。
However you can raise the sample size to 10 at the cost of some additional CPU usage in order to closely approximate true LRU, and check if this makes a difference in your cache misses rate.
你可以提升样品大小配置到10,它将接近真正的LRU算法,并且有不同错过率,但是要消耗更多的CPU。
To experiment in production with different values for the sample size by using the CONFIG SET maxmemory-samples <count>
command, is very simple.