企业级NoSql数据库Redis集群

一 关系型数据库和 NoSQL 数据库

1.1 数据库主要分为两大类：关系型数据库与 NoSQL 数据库

关系型数据库，是建立在关系模型基础上的数据库，其借助于集合代数等数学概念和方法来处理数据库 中的数据主流的 MySQL、Oracle、MS SQL Server 和 DB2 都属于这类传统数据库。

NoSQL 数据库，全称为 Not Only SQL，意思就是适用关系型数据库的时候就使用关系型数据库，不适 用的时候也没有必要非使用关系型数据库不可，可以考虑使用更加合适的数据存储。主要分为临时性键 值存储（memcached、Redis）、永久性键值存储（ROMA、Redis）、面向文档的数据库 （MongoDB、CouchDB）、面向列的数据库（Cassandra、HBase），每种 NoSQL 都有其特有的使用 场景及优点。

1.2 为什么还要用 NoSQL 数据库呢？

主要是由于随着互联网发展，数据量越来越大，对性能要求越来越高，传统数据库存在着先天性的缺 陷，即单机（单库）性能瓶颈，并且扩展困难。这样既有单机单库瓶颈，却又扩展困难，自然无法满足 日益增长的海量数据存储及其性能要求，所以才会出现了各种不同的 NoSQL 产品，NoSQL 根本性的优 势在于在云计算时代，简单、易于大规模分布式扩展，并且读写性能非常高

二 Remote Dictionary Server 简介

中文官网 https://redis.cn

2.1 什么是redis

Redis (Remote Dictionary Server)

在2009年发布，开发者是意大利的萨尔瓦多·桑菲利波普（Salvatore Sanfilippo），他本想为自己的公司 开发一个用于替换MySQL的产品Redis，但是没有想到他把Redis开源后大受欢迎，短短几年，Redis就有 了很大的用户群体，目前国内外使用的公司众多,比如:阿里,百度,新浪微博,知乎网,GitHub,Twitter 等

Redis是一个开源的、遵循BSD协议的、基于内存的而且目前比较流行的键值数据库(key-value database)，是一个非关系型数据库，redis 提供将内存通过网络远程共享的一种服务，提供类似功能的 还有memcached，但相比memcached，redis还提供了易扩展、高性能、具备数据持久性等功能。

Redis 在高并发、低延迟环境要求比较高的环境使用量非常广泛

2.2 Redis特性

速度快: 10W QPS,基于内存,C语言实现

单线程

持久化

支持多种数据结构

支持多种编程语言 功能丰富: 支持Lua脚本,发布订阅,事务,pipeline等功能

简单: 代码短小精悍(单机核心代码只有23000行左右),单线程开发容易,不依赖外部库,使用简单

主从复制

支持高可用和分布式

三 Redis的安装

官方下载地址：Index of /releases/

3.1 源码编译

#下载依赖包:
[root@redis-node1 ~]# dnf install make gcc initscripts  -y

#源码安装
[root@redis-node1 ~]# tar zxf redis-7.4.0.tar.gz
[root@redis-node1 ~]# ls
redis-7.4.0 redis-7.4.0.tar.gz

[root@redis-node1 ~]# make
[root@redis-node1 ~]# make install
[root@redis-node2 ~]# cd redis-7.4.0/
[root@redis-node2 redis-7.4.0]# cd utils/
[root@redis-node1 utils]# ./install_server.sh

Welcome to the redis service installer
This script will help you easily set up a running redis server
This systems seems to use systemd. #提示系统使用的是systemd的初始化方式

Please take a look at the provided example service unit files in this directory, 
and adapt and install them. Sorry!
[root@redis-node1 utils]# vim install_server.sh   
#解决报错问题  以下全部注释

#bail if this system is managed by systemd
#_pid_1_exe="$(readlink -f /proc/1/exe)"
#if [ "${_pid_1_exe##*/}" = systemd ]
#then
#       echo "This systems seems to use systemd."
#       echo "Please take a look at the provided example service unit files in 
this directory, and adapt and install them. Sorry!"
#       exit 1
#fi

[root@redis-node2 utils]# ./install_server.sh
Welcome to the redis service installer
This script will help you easily set up a running redis server

Please select the redis port for this instance: [6379]   #回车
Selecting default: 6379
Please select the redis config file name [/etc/redis/6379.conf]
Selected default - /etc/redis/6379.conf
Please select the redis log file name [/var/log/redis_6379.log]
Selected default - /var/log/redis_6379.log
Please select the data directory for this instance [/var/lib/redis/6379]
Selected default - /var/lib/redis/6379
Please select the redis executable path [/usr/local/bin/redis-server]
Selected config:
Port           : 6379
Config file    : /etc/redis/6379.conf
Log file       : /var/log/redis_6379.log
Data dir       : /var/lib/redis/6379
Executable     : /usr/local/bin/redis-server
Cli Executable : /usr/local/bin/redis-cli
Is this ok? Then press ENTER to go on or Ctrl-C to abort.
Copied /tmp/6379.conf => /etc/init.d/redis_6379
Installing service...
Successfully added to chkconfig!
Successfully added to runlevels 345!
Starting Redis server...
Installation successful!

#配置redis
[root@redis-node2 utils]# vim /etc/redis/6379.conf
改:bind  * -::*
改:protected-mode no

[root@redis-node2 utils]# /etc/init.d/redis_6379 restart
Stopping ...
Redis stopped
Starting Redis server...

#查看端口
[root@redis-node2 utils]# netstat -antlpe | grep redis
tcp        0      0 0.0.0.0:6379            0.0.0.0:*               LISTEN      0          63001      33998/redis-server
tcp6       0      0 :::6379                 :::*                    LISTEN      0          63002      33998/redis-server

[root@redis-node1 ~]# cd redis-7.4.0/
[root@redis-node1 redis-7.4.0]# /etc/init.d/redis_6379 restart
Stopping ...
Redis stopped
Starting Redis server...
[root@redis-node1 redis-7.4.0]# netstat -antlpe | grep redis
tcp        0      0 0.0.0.0:6379            0.0.0.0:*               LISTEN      0          26003      1805/redis-server *
tcp6       0      0 :::6379                 :::*                    LISTEN      0          26004      1805/redis-server *
[root@redis-node1 redis-7.4.0]# redis-cli
127.0.0.1:6379> info

#另外两台的配置:
[root@redis-node1 ~]# scp -r redis-7.4.0 root@172.25.254.20:/root/
[root@redis-node1 ~]# cd /usr/local/bin
[root@redis-node1 bin]# rsync -al * root@172.25.254.20:/usr/local/bin
以下步骤同上

四 Redis的基本操作

五 Redis 主从复制

5.1 环境配置

redis-node1 master

redis-node2 slave

redis-node3 slave

5.2 配置主从同步

1.修改mastser节点的配置文件

[root@redis-node2 utils]# vim /etc/redis/6379.conf
protected-mode no #关闭protected模式

[root@redis-node2 utils]# /etc/init.d/redis_6379 restart
Stopping ...
Redis stopped
Starting Redis server...

2.配置slave节点

[root@redis-node2 & 3 ~]# vim /etc/redis/6379.conf
replicaof 172.25.254.100 6379

[root@redis-node2 & 3 ~]# /etc/init.d/redis_6379 restart
Stopping ... 
Waiting for Redis to shutdown ... 
Redis stopped 
Starting Redis server...

3.测试效果

#在mastser节点
[root@redis-node1 ~]# redis-cli
127.0.0.1:6379> set name zyl 
OK

#在slave节点查看
[root@redis-node2/3 ~]# redis-cli
127.0.0.1:6379> get name
"zyl"

5.3 主从同步过程

slave节点发送同步亲求到master节点

slave节点通过master节点的认证开始进行同步

master节点会开启bgsave进程发送内存rbd到slave节点，在此过程中是异步操作，也就是说master节点仍然可以进行写入动作

slave节点收到rdb后首先清空自己的所有数据

slave节点加载rdb并进行数据恢复

在master和slave同步过程中master还会开启新的bgsave进程把没有同步的数据进行缓存

然后通过自有的replactionfeedslave函数把未通过内存快照发动到slave的数据一条一条写入到slave中

六 Redis的哨兵（高可用）

实验环境：我们用主两从来实现Redis的高可用架构

6.1 Redis哨兵

entinel 进程是用于监控redis集群中Master主服务器工作的状态，在Master主服务器发生故障的时候， 可以实现Master和Slave服务器的切换，保证系统的高可用，此功能在redis2.6+的版本已引用，Redis的 哨兵模式到了2.8版本之后就稳定了下来。一般在生产环境也建议使用Redis的2.8版本的以后版本 每个哨兵(Sentinel)进程会向其它哨兵(Sentinel)、Master、Slave定时发送消息，以确认对方是否”活”

着，如果发现对方在指定配置时间(此项可配置)内未得到回应，则暂时认为对方已离线，也就是所谓的”

主观认为宕机” (主观:是每个成员都具有的独自的而且可能相同也可能不同的意识)，英文名称：

Subjective Down，简称SDOWN

有主观宕机，对应的有客观宕机。当“哨兵群”中的多数Sentinel进程在对Master主服务器做出SDOWN 的 判断，并且通过 SENTINEL is-master-down-by-addr 命令互相交流之后，得出的Master Server下线判 断，这种方式就是“客观宕机”(客观:是不依赖于某种意识而已经实际存在的一切事物)，英文名称是：

Objectively Down， 简称 ODOWN

通过一定的vote算法，从剩下的slave从服务器节点中，选一台提升为Master服务器节点，然后自动修改 相关配置，并开启故障转移（failover）

Sentinel 机制可以解决master和slave角色的自动切换问题，但单个 Master 的性能瓶颈问题无法解决,类 似于MySQL中的MHA功能

Redis Sentinel中的Sentinel节点个数应该为大于等于3且最好为奇数

sentinel中的三个定时任务

每10秒每个sentinel对master和slave执行info

发现slave节点 确认主从关系 每2秒每个sentinel通过master节点的channel交换信息(pub/sub)

通过sentinel__:hello频道交互 交互对节点的“看法”和自身信息 每1秒每个sentinel对其他sentinel和redis执行pi

6.2 哨兵的实验过程

在所有阶段中关闭 protected-mode no

1.在master节点中

[root@redis-node1 ~]# cd redis-7.4.0/
[root@redis-node1 redis-7.4.0]# scp sentinel.conf sentinel.conf.bak
[root@redis-node1 redis-7.4.0]# ls
00-RELEASENOTES     INSTALL      redis.conf              runtest-sentinel   tests
BUGS                LICENSE.txt  REDISCONTRIBUTIONS.txt  SECURITY.md        TLS.md
CODE_OF_CONDUCT.md  Makefile     runtest                 sentinel.conf      utils
CONTRIBUTING.md     MANIFESTO    runtest-cluster         sentinel.conf.bak
deps                README.md    runtest-moduleapi       src
[root@redis-node1 redis-7.4.0]# cp sentinel.conf /etc/redis/
[root@redis-node1 redis-7.4.0]# vim /etc/redis/sentinel.conf
protected-mode no #关闭保护模式
port 26379 #监听端口
daemonize no #进入不打如后台
pidfile /var/run/redis-sentinel.pid #sentinel进程pid文件
loglevel notice #日志级别
sentinel monitor mymaster 172.25.254.100 6379 2 #创建sentinel监控监控master主机，2表示必须得到2票

sentinel down-after-milliseconds mymaster 10000 #master中断时长，10秒连不上视为master下线

sentinel parallel-syncs mymaster 1 #发生故障转移后，同时开始同步新master数据的slave数量

sentinel failover-timeout mymaster 180000 #整个故障切换的超时时间为3分钟


#复制配置文件到其他阶段:
[root@redis-node1 redis-7.4.0]# scp /etc/redis/sentinel.conf root@172.25.254.20:/etc/redis/
root@172.25.254.20's password:
sentinel.conf                                                      100%   14KB  15.7MB/s   00:00
[root@redis-node1 redis-7.4.0]# scp /etc/redis/sentinel.conf root@172.25.254.30:/etc/redis/
root@172.25.254.30's password:
sentinel.conf                                                      100%   14KB  18.4MB/s   00:00

2.启动服务

[root@redis-node1 redis-7.4.0]# redis-sentinel /etc/redis/sentinel.conf
7140:X 25 Aug 2024 15:55:41.872 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
7140:X 25 Aug 2024 15:55:41.872 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
7140:X 25 Aug 2024 15:55:41.873 * Redis version=7.4.0, bits=64, commit=00000000, modified=0, pid=7140, just started
7140:X 25 Aug 2024 15:55:41.873 * Configuration loaded
7140:X 25 Aug 2024 15:55:41.873 * Increased maximum number of open files to 10032 (it was originally set to 1024).
7140:X 25 Aug 2024 15:55:41.873 * monotonic clock: POSIX clock_gettime
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis Community Edition
  .-`` .-```.  ```\/    _.,_ ''-._     7.4.0 (00000000/0) 64 bit
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 26379
 |    `-._   `._    /     _.-'    |     PID: 7140
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           https://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

7140:X 25 Aug 2024 15:55:41.877 * Sentinel new configuration saved on disk
7140:X 25 Aug 2024 15:55:41.877 * Sentinel ID is 49e775c377a149747560f5c616579d8f80bd4cc4
7140:X 25 Aug 2024 15:55:41.878 # +monitor master mymaster 172.25.254.10 6379 quorum 2
7140:X 25 Aug 2024 15:55:41.879 * +slave slave 172.25.254.20:6379 172.25.254.20 6379 @ mymaster 172.25.254.10 6379
7140:X 25 Aug 2024 15:55:41.880 * Sentinel new configuration saved on disk
7140:X 25 Aug 2024 15:55:41.880 * +slave slave 172.25.254.30:6379 172.25.254.30 6379 @ mymaster 172.25.254.10 6379
7140:X 25 Aug 2024 15:55:41.880 * Sentinel new configuration saved on disk
^C7140:signal-handler (1724572589) Received SIGINT scheduling shutdown...
7140:X 25 Aug 2024 15:56:29.216 * User requested shutdown...
7140:X 25 Aug 2024 15:56:29.216 * Removing the pid file.
7140:X 25 Aug 2024 15:56:29.216 # Sentinel is now ready to exit, bye bye...

3.测试

127.0.0.1:6379> info replication

# Replication
role:slave
master_host:172.25.254.10
master_port:6379
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_read_repl_offset:6444
slave_repl_offset:6444
slave_priority:100
slave_read_only:1
replica_announced:1
connected_slaves:0
master_failover_state:no-failover
master_replid:2b1316e680524a7053d16f3a2d2b33ff77cc4b6b
master_replid2:db7dfdd2b72489cf055066683deafbc42a7f7d0f
master_repl_offset:6444
second_repl_offset:6183
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:15
repl_backlog_histlen:6430

127.0.0.1:6379> quit

6.3. 在整个架构中可能会出现的问题

问题： 在生产环境中如果master和slave中的网络出现故障，由于哨兵的存在会把master提出去 当网络恢复后，master发现环境发生改变，master就会把自己的身份转换成slave master变成slave后会把网络故障那段时间写入自己中的数据清掉，这样数据就丢失了。 解决：

master在被写入数据时会持续连接slave，mater确保有2个slave可以写入我才允许写入 如果slave数量少于2个便拒绝写入

七 Redis Cluster（无中心化设计）

7.1 Redis Cluster 工作原理

在哨兵sentinel机制中，可以解决redis高可用问题，即当master故障后可以自动将slave提升为master， 从而可以保证redis服务的正常使用，但是无法解决redis单机写入的瓶颈问题，即单机redis写入性能受 限于单机的内存大小、并发数量、网卡速率等因素。

redis 3.0版本之后推出了无中心架构的redis cluster机制，在无中心的redis集群当中，其每个节点保存 当前节点数据和整个集群状态,每个节点都和其他所有节点连接

Redis Cluster特点如下

1. 所有Redis节点使用(PING机制)互联

2. 集群中某个节点的是否失效，是由整个集群中超过半数的节点监测都失效，才能算真正的失效

3. 客户端不需要proxy即可直接连接redis，应用程序中需要配置有全部的redis服务器IP

4. redis cluster把所有的redis node 平均映射到 0-16383个槽位(slot)上，读写需要到指定的redis node上进行操作，因此有多少个redis node相当于redis 并发扩展了多少倍，每个redis node 承担,16384/N个槽位

5. Redis cluster预先分配16384个(slot)槽位，当需要在redis集群中写入一个key -value的时候，会使 用CRC16(key) mod 16384之后的值，决定将key写入值哪一个槽位从而决定写入哪一个Redis节点 上，从而有效解决单机瓶颈。

7.2部署redis cluster

在所有redis主机中

[root@redis-node1 ~]# yum install redis -y

[root@redis-node1 ~]# vim /etc/redis/redis.conf
masterauth "123456"            #集群主从认证
requirepass "123456"           #redis登陆密码 redis-cli 命令连接                                   redis后要用“auth 密码”进行认证
cluster-enabled yes            #开启cluster集群功能
cluster-config-file nodes-6379.conf #指定集群配置文件
cluster-node-timeout 15000     #节点加入集群的超时时间单位是ms

#重启
[root@redis-node1 ~]# systemctl enable --now redis
Created symlink /etc/systemd/system/multi-user.target.wants/redis.service → /usr/lib/systs.service.

#查看6379端口已开启
[root@redis-node1 ~]# netstat -antlupe | grep redis
tcp        0      0 0.0.0.0:6379            0.0.0.0:*               LISTEN      977      282/redis-server
tcp        0      0 0.0.0.0:16379           0.0.0.0:*               LISTEN      977      282/redis-server
tcp6       0      0 :::6379                 :::*                    LISTEN      977      282/redis-server
tcp6       0      0 :::16379                :::*                    LISTEN      977      282/redis-server

#复制到其他node
[root@redis-node1 ~]# for i in 20 30 110 120 130

> do
> scp /etc/redis/redis.conf root@172.25.254.$i:/etc/redis/redis.conf
> done

#所有redis重启
[root@redis-node1 ~]# systemctl systemctl enable  --now redis


#node1中无法写入
[root@redis-node1 ~]# redis-cli
127.0.0.1:6379> keys *
(error) NOAUTH Authentication required.
127.0.0.1:6379> auth 123456
OK
127.0.0.1:6379> keys *
(empty array)
127.0.0.1:6379> set name wang
(error) CLUSTERDOWN Hash slot not served
127.0.0.1:6379> quit

7.3创建redis-cluster

#创建
[root@redis-node1 ~]# redis-cli --cluster create -a 123456 \

> 172.25.254.10:6379 172.25.254.20:6379 172.25.254.30:6379 \
> 172.25.254.110:6379 172.25.254.120:6379 172.25.254.130:6379 \
> --cluster-replicas 1
> Warning: Using a password with '-a' or '-u' option on the command line interface may not
>
> >> Performing hash slots allocation on 6 nodes...
> >> Master[0] -> Slots 0 - 5460
> >> Master[1] -> Slots 5461 - 10922
> >> Master[2] -> Slots 10923 - 16383
> >> Adding replica 172.25.254.120:6379 to 172.25.254.10:6379
> >> Adding replica 172.25.254.130:6379 to 172.25.254.20:6379
> >> Adding replica 172.25.254.110:6379 to 172.25.254.30:6379
> >> M: c7a8b0d93c302e0285acdf93cee4d69cce614d90 172.25.254.10:6379
> >> slots:[0-5460] (5461 slots) master
> >> M: 3eba692261902cf9d4ecbe22e62dceff02b1c6ea 172.25.254.20:6379
> >> slots:[5461-10922] (5462 slots) master
> >> M: a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08 172.25.254.30:6379
> >> slots:[10923-16383] (5461 slots) master
> >> S: 86f23467487b7f1e1985264bc27ebd3de02e7f5f 172.25.254.110:6379
> >> replicates a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08
> >> S: d9a4b0d1a92c15224773ff937aa7783ca8a8b56d 172.25.254.120:6379
> >> replicates c7a8b0d93c302e0285acdf93cee4d69cce614d90
> >> S: bb5bbc578128be4c4d4975484d94a472ecb87302 172.25.254.130:6379
> >> replicates 3eba692261902cf9d4ecbe22e62dceff02b1c6ea
> >> Can I set the above configuration? (type 'yes' to accept): yes
> >> Nodes configuration updated
> >> Assign a different config epoch to each node
> >> Sending CLUSTER MEET messages to join the cluster
> >> Waiting for the cluster to join
> >> .
> >> Performing Cluster Check (using node 172.25.254.10:6379)
> >> M: c7a8b0d93c302e0285acdf93cee4d69cce614d90 172.25.254.10:6379
> >> slots:[0-5460] (5461 slots) master
> >> 1 additional replica(s)
> >> S: d9a4b0d1a92c15224773ff937aa7783ca8a8b56d 172.25.254.120:6379
> >> slots: (0 slots) slave
> >> replicates c7a8b0d93c302e0285acdf93cee4d69cce614d90
> >> M: a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08 172.25.254.30:6379
> >> slots:[10923-16383] (5461 slots) master
> >> 1 additional replica(s)
> >> S: 86f23467487b7f1e1985264bc27ebd3de02e7f5f 172.25.254.110:6379
> >> slots: (0 slots) slave
> >> replicates a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08
> >> M: 3eba692261902cf9d4ecbe22e62dceff02b1c6ea 172.25.254.20:6379
> >> slots:[5461-10922] (5462 slots) master
> >> 1 additional replica(s)
> >> S: bb5bbc578128be4c4d4975484d94a472ecb87302 172.25.254.130:6379
> >> slots: (0 slots) slave
> >> replicates 3eba692261902cf9d4ecbe22e62dceff02b1c6ea
> >> [OK] All nodes agree about slots configuration.
> >> Check for open slots...
> >> Check slots coverage...
> >> [OK] All 16384 slots covered.

检测redis集群状态

[root@redis-node1 ~]# redis-cli -a 123456 --cluster info 172.25.254.10:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not
172.25.254.10:6379 (c7a8b0d9...) -> 0 keys | 5461 slots | 1 slaves.
172.25.254.30:6379 (a8ecbc9c...) -> 0 keys | 5461 slots | 1 slaves.
172.25.254.20:6379 (3eba6922...) -> 0 keys | 5462 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.

#检测集群
[root@redis-node1 ~]# redis-cli -a 123456 --cluster check 172.25.254.10:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not
172.25.254.10:6379 (c7a8b0d9...) -> 0 keys | 5461 slots | 1 slaves.
172.25.254.30:6379 (a8ecbc9c...) -> 0 keys | 5461 slots | 1 slaves.
172.25.254.20:6379 (3eba6922...) -> 0 keys | 5462 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 172.25.254.10:6379)
M: c7a8b0d93c302e0285acdf93cee4d69cce614d90 172.25.254.10:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: d9a4b0d1a92c15224773ff937aa7783ca8a8b56d 172.25.254.120:6379
   slots: (0 slots) slave
   replicates c7a8b0d93c302e0285acdf93cee4d69cce614d90
M: a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08 172.25.254.30:6379
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 86f23467487b7f1e1985264bc27ebd3de02e7f5f 172.25.254.110:6379
   slots: (0 slots) slave
   replicates a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08
M: 3eba692261902cf9d4ecbe22e62dceff02b1c6ea 172.25.254.20:6379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: bb5bbc578128be4c4d4975484d94a472ecb87302 172.25.254.130:6379
   slots: (0 slots) slave
   replicates 3eba692261902cf9d4ecbe22e62dceff02b1c6ea
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

写入数据

[root@redis-node1 ~]# redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not
127.0.0.1:6379> set name wang
(error) MOVED 5798 172.25.254.20:6379   #分配到20的hash槽
127.0.0.1:6379> get name
(error) MOVED 5798 172.25.254.20:6379    
127.0.0.1:6379>

#只能在20上写入和读取
[root@redis-node2 ~]# redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not
127.0.0.1:6379> set name wang
OK
127.0.0.1:6379> get name
"wang"
127.0.0.1:6379>

7.4 集群扩容

添加master

[root@redis-node1 ~]# scp /etc/redis/redis.conf root@172.25.254.50:/etc/redis/redis.conf
[root@redis-node1 ~]# scp /etc/redis/redis.conf root@172.25.254.150:/etc/redis/redis.conf

#添加master
[root@redis-node1 ~]# redis-cli -a 123456 --cluster add-node 172.25.254.50:6379 172.25.254.10:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.

>>> Adding node 172.25.254.50:6379 to cluster 172.25.254.10:6379
>>> Performing Cluster Check (using node 172.25.254.10:6379)
>>> M: c7a8b0d93c302e0285acdf93cee4d69cce614d90 172.25.254.10:6379
>>> slots:[0-5460] (5461 slots) master
>>> 1 additional replica(s)
>>> S: d9a4b0d1a92c15224773ff937aa7783ca8a8b56d 172.25.254.120:6379
>>> slots: (0 slots) slave
>>> replicates c7a8b0d93c302e0285acdf93cee4d69cce614d90
>>> M: a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08 172.25.254.30:6379
>>> slots:[10923-16383] (5461 slots) master
>>> 1 additional replica(s)
>>> S: 86f23467487b7f1e1985264bc27ebd3de02e7f5f 172.25.254.110:6379
>>> slots: (0 slots) slave
>>> replicates a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08
>>> M: 3eba692261902cf9d4ecbe22e62dceff02b1c6ea 172.25.254.20:6379
>>> slots:[5461-10922] (5462 slots) master
>>> 1 additional replica(s)
>>> S: bb5bbc578128be4c4d4975484d94a472ecb87302 172.25.254.130:6379
>>> slots: (0 slots) slave
>>> replicates 3eba692261902cf9d4ecbe22e62dceff02b1c6ea
>>> [OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
>>> [OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 172.25.254.50:6379 to make it join the cluster.
>>> [OK] New node added correctly.


#查看
[root@redis-node1 ~]# redis-cli -a 123456 --cluster check 172.25.254.10:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.25.254.10:6379 (c7a8b0d9...) -> 0 keys | 5461 slots | 1 slaves.
172.25.254.30:6379 (a8ecbc9c...) -> 0 keys | 5461 slots | 1 slaves.
172.25.254.50:6379 (56986691...) -> 0 keys | 0 slots | 0 slaves.
172.25.254.20:6379 (3eba6922...) -> 1 keys | 5462 slots | 1 slaves.
[OK] 1 keys in 4 masters.
0.00 keys per slot on average.

>>> Performing Cluster Check (using node 172.25.254.10:6379)
>>> M: c7a8b0d93c302e0285acdf93cee4d69cce614d90 172.25.254.10:6379
>>> slots:[0-5460] (5461 slots) master
>>> 1 additional replica(s)
>>> S: d9a4b0d1a92c15224773ff937aa7783ca8a8b56d 172.25.254.120:6379
>>> slots: (0 slots) slave
>>> replicates c7a8b0d93c302e0285acdf93cee4d69cce614d90
>>> M: a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08 172.25.254.30:6379
>>> slots:[10923-16383] (5461 slots) master
>>> 1 additional replica(s)
>>> S: 86f23467487b7f1e1985264bc27ebd3de02e7f5f 172.25.254.110:6379
>>> slots: (0 slots) slave
>>> replicates a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08
>>> M: 569866915eebbb86b2d59deda26f7c3467627ddf 172.25.254.50:6379
>>> slots: (0 slots) master
>>> M: 3eba692261902cf9d4ecbe22e62dceff02b1c6ea 172.25.254.20:6379
>>> slots:[5461-10922] (5462 slots) master
>>> 1 additional replica(s)
>>> S: bb5bbc578128be4c4d4975484d94a472ecb87302 172.25.254.130:6379
>>> slots: (0 slots) slave
>>> replicates 3eba692261902cf9d4ecbe22e62dceff02b1c6ea
>>> [OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
>>> [OK] All 16384 slots covered.

#查看
[root@redis-node1 ~]# redis-cli -a 123456 --cluster check 172.25.254.10:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.25.254.10:6379 (c7a8b0d9...) -> 0 keys | 5461 slots | 1 slaves.
172.25.254.30:6379 (a8ecbc9c...) -> 0 keys | 5461 slots | 1 slaves.
172.25.254.50:6379 (56986691...) -> 0 keys | 0 slots | 0 slaves.
172.25.254.20:6379 (3eba6922...) -> 1 keys | 5462 slots | 1 slaves.
[OK] 1 keys in 4 masters.
0.00 keys per slot on average.

>>> Performing Cluster Check (using node 172.25.254.10:6379)
>>> M: c7a8b0d93c302e0285acdf93cee4d69cce614d90 172.25.254.10:6379
>>> slots:[0-5460] (5461 slots) master
>>> 1 additional replica(s)
>>> S: d9a4b0d1a92c15224773ff937aa7783ca8a8b56d 172.25.254.120:6379
>>> slots: (0 slots) slave
>>> replicates c7a8b0d93c302e0285acdf93cee4d69cce614d90
>>> M: a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08 172.25.254.30:6379
>>> slots:[10923-16383] (5461 slots) master
>>> 1 additional replica(s)
>>> S: 86f23467487b7f1e1985264bc27ebd3de02e7f5f 172.25.254.110:6379
>>> slots: (0 slots) slave
>>> replicates a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08
>>> M: 569866915eebbb86b2d59deda26f7c3467627ddf 172.25.254.50:6379
>>> slots: (0 slots) master
>>> M: 3eba692261902cf9d4ecbe22e62dceff02b1c6ea 172.25.254.20:6379
>>> slots:[5461-10922] (5462 slots) master
>>> 1 additional replica(s)
>>> S: bb5bbc578128be4c4d4975484d94a472ecb87302 172.25.254.130:6379
>>> slots: (0 slots) slave
>>> replicates 3eba692261902cf9d4ecbe22e62dceff02b1c6ea
>>> [OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
>>> [OK] All 16384 slots covered.

分配槽位

[root@redis-node1 ~]# redis-cli -a 123456 --cluster reshard  172.25.254.10:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.

>>> Performing Cluster Check (using node 172.25.254.10:6379)
>>> M: c7a8b0d93c302e0285acdf93cee4d69cce614d90 172.25.254.10:6379
>>> slots:[0-5460] (5461 slots) master
>>> 1 additional replica(s)
>>> S: d9a4b0d1a92c15224773ff937aa7783ca8a8b56d 172.25.254.120:6379
>>> slots: (0 slots) slave
>>> replicates c7a8b0d93c302e0285acdf93cee4d69cce614d90
>>> M: a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08 172.25.254.30:6379
>>> slots:[10923-16383] (5461 slots) master
>>> 1 additional replica(s)
>>> S: 86f23467487b7f1e1985264bc27ebd3de02e7f5f 172.25.254.110:6379
>>> slots: (0 slots) slave
>>> replicates a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08
>>> M: 569866915eebbb86b2d59deda26f7c3467627ddf 172.25.254.50:6379
>>> slots: (0 slots) master
>>> M: 3eba692261902cf9d4ecbe22e62dceff02b1c6ea 172.25.254.20:6379
>>> slots:[5461-10922] (5462 slots) master
>>> 1 additional replica(s)
>>> S: bb5bbc578128be4c4d4975484d94a472ecb87302 172.25.254.130:6379
>>> slots: (0 slots) slave
>>> replicates 3eba692261902cf9d4ecbe22e62dceff02b1c6ea
>>> [OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
>>> [OK] All 16384 slots covered.
>>> How many slots do you want to move (from 1 to 16384)? 4096
>>> What is the receiving node ID?     569866915eebbb86b2d59deda26f7c3467627ddf    #给50分配槽位，50ID
>>> Please enter all the source node IDs.
>>> Type 'all' to use all the nodes as source nodes for the hash slots.
>>> Type 'done' once you entered all the source nodes IDs.
>>> Source node #1: all            #让所有给他分配

添加slave

[root@redis-node1 ~]# redis-cli -a 123456 --cluster add-node 172.25.254.150:6379 172.25.254.10:6379 --cluster-slave --cluster-master-id 569866915eebbb86b2d59deda26f7c3467627ddf
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.

>>> Adding node 172.25.254.150:6379 to cluster 172.25.254.10:6379
>>> Performing Cluster Check (using node 172.25.254.10:6379)
>>> M: c7a8b0d93c302e0285acdf93cee4d69cce614d90 172.25.254.10:6379
>>> slots:[1365-5460] (4096 slots) master
>>> 1 additional replica(s)
>>> S: d9a4b0d1a92c15224773ff937aa7783ca8a8b56d 172.25.254.120:6379
>>> slots: (0 slots) slave
>>> replicates c7a8b0d93c302e0285acdf93cee4d69cce614d90
>>> M: a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08 172.25.254.30:6379
>>> slots:[12288-16383] (4096 slots) master
>>> 1 additional replica(s)
>>> S: 86f23467487b7f1e1985264bc27ebd3de02e7f5f 172.25.254.110:6379
>>> slots: (0 slots) slave
>>> replicates a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08
>>> M: 569866915eebbb86b2d59deda26f7c3467627ddf 172.25.254.50:6379
>>> slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
>>> M: 3eba692261902cf9d4ecbe22e62dceff02b1c6ea 172.25.254.20:6379
>>> slots:[6827-10922] (4096 slots) master
>>> 1 additional replica(s)
>>> S: bb5bbc578128be4c4d4975484d94a472ecb87302 172.25.254.130:6379
>>> slots: (0 slots) slave
>>> replicates 3eba692261902cf9d4ecbe22e62dceff02b1c6ea
>>> [OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
>>> [OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 172.25.254.150:6379 to make it join the cluster.
>>> Waiting for the cluster to join

>>> Configure node as replica of 172.25.254.50:6379.
>>> [OK] New node added correctly.


#查看
[root@redis-node1 ~]# redis-cli -a 123456 --cluster check 172.25.254.10:6379             
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.25.254.10:6379 (c7a8b0d9...) -> 0 keys | 4096 slots | 1 slaves.
172.25.254.30:6379 (a8ecbc9c...) -> 0 keys | 4096 slots | 1 slaves.
172.25.254.50:6379 (56986691...) -> 1 keys | 4096 slots | 1 slaves.
172.25.254.20:6379 (3eba6922...) -> 0 keys | 4096 slots | 1 slaves.
[OK] 1 keys in 4 masters.
0.00 keys per slot on average.

>>> Performing Cluster Check (using node 172.25.254.10:6379)
>>> M: c7a8b0d93c302e0285acdf93cee4d69cce614d90 172.25.254.10:6379
>>> slots:[1365-5460] (4096 slots) master
>>> 1 additional replica(s)
>>> S: d9a4b0d1a92c15224773ff937aa7783ca8a8b56d 172.25.254.120:6379
>>> slots: (0 slots) slave
>>> replicates c7a8b0d93c302e0285acdf93cee4d69cce614d90
>>> M: a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08 172.25.254.30:6379
>>> slots:[12288-16383] (4096 slots) master
>>> 1 additional replica(s)
>>> S: 4ce9a070db536e0d8a85527928753e9ce96b1e1a 172.25.254.150:6379
>>> slots: (0 slots) slave
>>> replicates 569866915eebbb86b2d59deda26f7c3467627ddf
>>> S: 86f23467487b7f1e1985264bc27ebd3de02e7f5f 172.25.254.110:6379
>>> slots: (0 slots) slave
>>> replicates a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08
>>> M: 569866915eebbb86b2d59deda26f7c3467627ddf 172.25.254.50:6379
>>> slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
>>> 1 additional replica(s)
>>> M: 3eba692261902cf9d4ecbe22e62dceff02b1c6ea 172.25.254.20:6379
>>> slots:[6827-10922] (4096 slots) master
>>> 1 additional replica(s)
>>> S: bb5bbc578128be4c4d4975484d94a472ecb87302 172.25.254.130:6379
>>> slots: (0 slots) slave
>>> replicates 3eba692261902cf9d4ecbe22e62dceff02b1c6ea
>>> [OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
>>> [OK] All 16384 slots covered.

7.5 cluster集群维护

添加节点的时候是先添加node节点到集群，然后分配槽位，删除节点的操作与添加节点的操作正好相 反，是先将被删除的Redis node上的槽位迁移到集群中的其他Redis node节点上，然后再将其删除，如 果一个Redis node节点上的槽位没有被完全迁移，删除该node的时候会提示有数据且无法删除。

删除slave

#删除slave
[root@redis-node1 ~]# redis-cli -a 123456 --cluster del-node 172.25.254.150:6379  4ce9a070db536e0d8a85527928753e9ce96b1e1a
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.

>>> Removing node 4ce9a070db536e0d8a85527928753e9ce96b1e1a from cluster 172.25.254.150:6379
>>> Sending CLUSTER FORGET messages to the cluster...
>>> Sending CLUSTER RESET SOFT to the deleted node.


#查看删除成功
[root@redis-node1 ~]# redis-cli -a 123456 --cluster check 172.25.254.10:6379             Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.25.254.10:6379 (c7a8b0d9...) -> 0 keys | 4096 slots | 1 slaves.
172.25.254.30:6379 (a8ecbc9c...) -> 0 keys | 4096 slots | 1 slaves.
172.25.254.50:6379 (56986691...) -> 1 keys | 4096 slots | 0 slaves.
172.25.254.20:6379 (3eba6922...) -> 0 keys | 4096 slots | 1 slaves.
[OK] 1 keys in 4 masters.
0.00 keys per slot on average.

>>> Performing Cluster Check (using node 172.25.254.10:6379)
>>> M: c7a8b0d93c302e0285acdf93cee4d69cce614d90 172.25.254.10:6379
>>> slots:[1365-5460] (4096 slots) master
>>> 1 additional replica(s)
>>> S: d9a4b0d1a92c15224773ff937aa7783ca8a8b56d 172.25.254.120:6379
>>> slots: (0 slots) slave
>>> replicates c7a8b0d93c302e0285acdf93cee4d69cce614d90
>>> M: a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08 172.25.254.30:6379
>>> slots:[12288-16383] (4096 slots) master
>>> 1 additional replica(s)
>>> S: 86f23467487b7f1e1985264bc27ebd3de02e7f5f 172.25.254.110:6379
>>> slots: (0 slots) slave
>>> replicates a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08
>>> M: 569866915eebbb86b2d59deda26f7c3467627ddf 172.25.254.50:6379
>>> slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
>>> M: 3eba692261902cf9d4ecbe22e62dceff02b1c6ea 172.25.254.20:6379
>>> slots:[6827-10922] (4096 slots) master
>>> 1 additional replica(s)
>>> S: bb5bbc578128be4c4d4975484d94a472ecb87302 172.25.254.130:6379
>>> slots: (0 slots) slave
>>> replicates 3eba692261902cf9d4ecbe22e62dceff02b1c6ea
>>> [OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
>>> [OK] All 16384 slots covered.

移除槽位

[root@redis-node1 ~]# redis-cli -a 123456 --cluster reshard 172.25.254.10:6379

删除master

[root@redis-node1 ~]# redis-cli -a 123456 --cluster del-node 172.25.254.50:6379 569866915eebbb86b2d59deda26f7c3467627ddf
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.

>>> Removing node 569866915eebbb86b2d59deda26f7c3467627ddf from cluster 172.25.254.50:6379
>>> Sending CLUSTER FORGET messages to the cluster...
>>> Sending CLUSTER RESET SOFT to the deleted node.


#查看，已全部删除
[root@redis-node1 ~]# redis-cli -a 123456 --cluster check 172.25.254.10:6379             Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.25.254.10:6379 (c7a8b0d9...) -> 1 keys | 8192 slots | 1 slaves.
172.25.254.30:6379 (a8ecbc9c...) -> 0 keys | 4096 slots | 1 slaves.
172.25.254.20:6379 (3eba6922...) -> 0 keys | 4096 slots | 1 slaves.
[OK] 1 keys in 3 masters.
0.00 keys per slot on average.

>>> Performing Cluster Check (using node 172.25.254.10:6379)
>>> M: c7a8b0d93c302e0285acdf93cee4d69cce614d90 172.25.254.10:6379
>>> slots:[0-6826],[10923-12287] (8192 slots) master
>>> 1 additional replica(s)
>>> S: d9a4b0d1a92c15224773ff937aa7783ca8a8b56d 172.25.254.120:6379
>>> slots: (0 slots) slave
>>> replicates c7a8b0d93c302e0285acdf93cee4d69cce614d90
>>> M: a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08 172.25.254.30:6379
>>> slots:[12288-16383] (4096 slots) master
>>> 1 additional replica(s)
>>> S: 86f23467487b7f1e1985264bc27ebd3de02e7f5f 172.25.254.110:6379
>>> slots: (0 slots) slave
>>> replicates a8ecbc9c54bdd70e1f68b6c02e1ce5d4b907ff08
>>> M: 3eba692261902cf9d4ecbe22e62dceff02b1c6ea 172.25.254.20:6379
>>> slots:[6827-10922] (4096 slots) master
>>> 1 additional replica(s)
>>> S: bb5bbc578128be4c4d4975484d94a472ecb87302 172.25.254.130:6379
>>> slots: (0 slots) slave
>>> replicates 3eba692261902cf9d4ecbe22e62dceff02b1c6ea
>>> [OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
>>> [OK] All 16384 slots covered.