Redis集群

一、集群简介

现状问题

业务发展过程中遇到的峰值瓶颈

• redis提供的服务OPS可以达到10万/秒，当前业务OPS已经达到10万/秒
• 内存单机容量达到256G，当前业务需求内存容量1T

使用集群的方式可以快速解决上述问题

集群架构

集群就是使用网络将若干台计算机联通起来，并提供统一的管理方式，使其对外呈现单机的服务效果

集群作用

• 分散单台服务器的访问压力，实现负载均衡
• 分散单台服务器的存储压力，实现可扩展性
• 降低单台服务器宕机带来的业务灾难

二、Redis集群结构设计

数据存储设计

• 通过算法设计，计算出key应该保存的位置
• 将所有的存储空间计划切割成16384份，每台主机保存一部分
  每份代表的是一个存储空间，不是一个key的保存空间
• 将key按照计算出的结果放到对应的存储空间

问题：如果集群中有一个节点宕机或者集群中又加入一台新的节点呢？

• 增强可扩展性

集群内部通讯设计

• 各个数据库相互通信，保存各个库中槽的编号数据（槽用来区分数据的存储空间位置）
• 一次命中，直接返回
• 一次未命中，告知具体位置

三、cluster集群结构搭建

配置项说明：

cluster-enabled yes|no		# 添加节点
cluster-config-file <filename>		#  cluster配置文件名，该文件属于自动生成，仅用于快速查找文件并查询文件内容
cluster-node-timeout <milliseconds>		# 节点服务响应超时时间，用于判定该节点是否下线或切换为从节点
cluster-migration-barrier <count>		# master连接的slave最小数量

节点操作命令：

cluster nodes		# 查看集群节点信息
cluster replicate <master-id>		# 进入一个从节点 redis，切换其主节点\
cluster meet ip:port		# 发现一个新节点，新增主节点
cluster forget <id>			# 忽略一个没有solt的节点
cluster failover		# 手动故障转移

redis-trib命令:

redis-trib.rb add-node		# 添加节点
redis-trib.rb del-node		# 删除节点
redis-trib.rb reshard		# 重新分片

配置集群为3主3从，使用一台机器的不同端口模拟3主3从进行

[root@redis-master ~]# vim /usr/local/redis/redis-4.0.0/conf/redis-6379.conf
port 6379
daemonize no
dir /usr/local/redis/redis-4.0.0/data
dbfilename dump-6379.rdb
rdbcompression yes
rdbchecksum yes
appendfilename appendonly-6379.aof
cluster-enabled yes
cluster-config-file nodes-6379.conf
cluster-node-timeout 10000
[root@redis-master ~]# redis-server /usr/local/redis/redis-4.0.0/conf/redis-6379.conf
38114:C 31 Aug 18:44:18.078 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
38114:C 31 Aug 18:44:18.078 # Redis version=4.0.0, bits=64, commit=00000000, modified=0, pid=38114, just started
38114:C 31 Aug 18:44:18.078 # Configuration loaded
38114:M 31 Aug 18:44:18.078 * Increased maximum number of open files to 10032 (it was originally set to 1024).
38114:M 31 Aug 18:44:18.079 * No cluster configuration found, I'm 1a25e01038a516d023e3286180e2a5641e1e71a2
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 4.0.0 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in cluster mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 38114
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

38114:M 31 Aug 18:44:18.081 # Server initialized
38114:M 31 Aug 18:44:18.081 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
38114:M 31 Aug 18:44:18.081 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
38114:M 31 Aug 18:44:18.081 * Ready to accept connections

[root@redis-master ~]# vim /usr/local/redis/redis-4.0.0/conf/redis-6380.conf
port 6380
daemonize no
dir /usr/local/redis/redis-4.0.0/data
dbfilename dump-6380.rdb
rdbcompression yes
rdbchecksum yes
appendfilename appendonly-6380.aof
cluster-enabled yes
cluster-config-file nodes-6380.conf
cluster-node-timeout 10000
[root@redis-master ~]# redis-server /usr/local/redis/redis-4.0.0/conf/redis-6380.conf
38118:C 31 Aug 18:44:19.890 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
38118:C 31 Aug 18:44:19.890 # Redis version=4.0.0, bits=64, commit=00000000, modified=0, pid=38118, just started
38118:C 31 Aug 18:44:19.890 # Configuration loaded
38118:M 31 Aug 18:44:19.890 * Increased maximum number of open files to 10032 (it was originally set to 1024).
38118:M 31 Aug 18:44:19.892 * No cluster configuration found, I'm 4aa775e66e0e3e20862fe1f089468815267fa9a6
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 4.0.0 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in cluster mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6380
 |    `-._   `._    /     _.-'    |     PID: 38118
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

38118:M 31 Aug 18:44:19.893 # Server initialized
38118:M 31 Aug 18:44:19.893 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
38118:M 31 Aug 18:44:19.893 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
38118:M 31 Aug 18:44:19.893 * Ready to accept connections

[root@redis-master ~]# vim /usr/local/redis/redis-4.0.0/conf/redis-6381.conf
port 6381
daemonize no
dir /usr/local/redis/redis-4.0.0/data
dbfilename dump-6381.rdb
rdbcompression yes
rdbchecksum yes
appendfilename appendonly-6381.aof
cluster-enabled yes
cluster-config-file nodes-6381.conf
cluster-node-timeout 10000
[root@redis-master ~]# redis-server /usr/local/redis/redis-4.0.0/conf/redis-6381.conf
38122:C 31 Aug 18:44:20.805 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
38122:C 31 Aug 18:44:20.805 # Redis vers