RocketMQ因缓存页忙导致Producer报Rejected，以及相关参数配置

 

RMQ以下逻辑会触发Rejected

 

 

 

 

相关参数：os相关：dirty_background_ratio、dirty_ratio ， RMQ相关：osPageCacheBusyTimeoutMills

https://sites.google.com/site/mytechnotesproject/home/announce/vmdirtyratioandvmdirtybackgroundratio

vm.dirty_background_ratio:这个参数指定了当文件系统缓存脏页数量达到系统内存百分之多少时（如5%）就会触发pdflush/flush/kdmflush等后台回写进程运行，将一定缓存的脏页异步地刷入外存；

vm.dirty_ratio:而这个参数则指定了当文件系统缓存脏页数量达到系统内存百分之多少时（如10%），系统不得不开始处理缓存脏页（因为此时脏页数量已经比较多，为了避免数据丢失需要将一定脏页刷入外存）；在此过程中很多应用进程可能会因为系统转而处理文件IO而阻塞。

vm.dirty_expire_centisecs is how long something can be in cache before it needs to be written. 默认30s， When the pdflush/flush/kdmflush processes kick in they will check to see how old a dirty page is, and if it’s older than this value it’ll be written asynchronously to disk. Since holding a dirty page in memory is unsafe this is also a safeguard against data loss.

vm.dirty_writeback_centisecs is how often the pdflush/flush/kdmflush processes wake up and check to see if work needs to be done.

不同情景的最佳实践

1.where the data contained on a Linux guest isn’t critical and can be lost, and usually where an application is writing to the same files repeatedly or in repeatable bursts. In theory, by allowing more dirty pages to exist in memory you’ll rewrite the same blocks over and over in cache, and just need to do one write every so often to the actual disk. To do this we raise the parameters:   vm.dirty_background_ratio = 50 vm.dirty_ratio = 80

2.There are also scenarios where a system has to deal with infrequent, bursty traffic to slow disk (batch jobs at the top of the hour, midnight, writing to an SD card on a Raspberry Pi, etc.). vm.dirty_background_ratio = 5 vm.dirty_ratio = 80

3.大数据量持续输入：vm.dirty_background_ratio = 5 vm.dirty_ratio = 10

 

压测结果 https://download.csdn.net/download/u012364631/11084548