Redis多机多实例集群部署

Redis多机多实例集群部署

安装redis单节点
官网下载二进制源码包安装，包里面为预编译过的，解压后可以直接二进制安装：

第一台虚拟机操作

第二台虚拟机跟第一台操作相同，只需要修改端口和修改节点，如果用多台虚拟机也是如此

进入/usr/local/下载gcc gcc-c++

[root@ c7-41 local]# yum -y install gcc gcc-c++
[root@ c7-41 7002]# systemctl stop firewalld	##注意关闭防火墙
[root@ c7-41 7002]# setenforce 0
setenforce: SELinux is disabled

下载redis包并解压安装

[root@ c7-41 local]# tar -zxvf redis-5.0.4.tar.gz
[root@ c7-41 local]# cd redis-5.0.4/
[root@ c7-41 redis-5.0.4]# make && make install

创建一个目录并在目录下创建节点

[root@ c7-41 redis-5.0.4]# mkdir redis-cluster
[root@ c7-41 redis-5.0.4]# cd redis-cluster/
[root@ c7-41 redis-cluster]# mkdir {7000..7002}
[root@ c7-41 redis-cluster]# ls
7000  7001  7002

修改配置文件并启动redis
一般需要把daemonize no 改为 daemonize yes，bind后的IP改为0.0.0.0

[root@ c7-41 redis-cluster]# cd ..
[root@ c7-41 redis-5.0.4]# vim redis.conf
daemonize yes	
bind 0.0.0.0
[root@ c7-41 redis-5.0.4]# redis-server redis.conf
7358:C 01 Apr 2021 19:33:33.432 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
7358:C 01 Apr 2021 19:33:33.432 # Redis version=5.0.4, bits=64, commit=00000000, modified=0, pid=7358, just started
7358:C 01 Apr 2021 19:33:33.432 # Configuration loaded
##表示启动成功

在刚才的每个节点目录里创建redis.conf配置文件，以7000节点为例：剩下的每个端口号都需要修改

[root@ c7-41 redis-5.0.4]# cd redis-cluster/
[root@ c7-41 redis-cluster]# ls
7000  7001  7002
[root@ c7-41 redis-cluster]# cd 7000
[root@ c7-41 7000]# vim redis.conf
port 7000
bind 0.0.0.0
protected-mode no
daemonize yes
pidfile /usr/local/redis-cluster/7000/redis_7000.pid
cluster-enabled yes
cluster-config-file nodes_7000.conf
cluster-node-timeout 15000
appendonly  yes

启动redis主节点和其余节点：（启动多实例）

[root@ c7-41 redis-cluster]# cd 7000
[root@ c7-41 7000]# redis-server redis.conf
7387:C 01 Apr 2021 19:42:46.247 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
7387:C 01 Apr 2021 19:42:46.247 # Redis version=5.0.4, bits=64, commit=00000000, modified=0, pid=7387, just started
7387:C 01 Apr 2021 19:42:46.247 # Configuration loaded
[root@ c7-41 7000]# cd ../7001/
[root@ c7-41 7001]# redis-server redis.conf
7392:C 01 Apr 2021 19:42:56.976 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
7392:C 01 Apr 2021 19:42:56.976 # Redis version=5.0.4, bits=64, commit=00000000, modified=0, pid=7392, just started
7392:C 01 Apr 2021 19:42:56.976 # Configuration loaded
[root@ c7-41 7001]# cd ../7002/
[root@ c7-41 7002]# redis-server redis.conf
7397:C 01 Apr 2021 19:43:03.703 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
7397:C 01 Apr 2021 19:43:03.703 # Redis version=5.0.4, bits=64, commit=00000000, modified=0, pid=7397, just started
7397:C 01 Apr 2021 19:43:03.703 # Configuration loaded

检查节点状态：

[root@ c7-41 7002]# ps -ef |grep redis
root       7359      1  0 19:33 ?        00:00:00 redis-server 0.0.0.0:6379
root       7388      1  0 19:42 ?        00:00:00 redis-server 0.0.0.0:7000 [cluster]
root       7393      1  0 19:42 ?        00:00:00 redis-server 0.0.0.0:7001 [cluster]
root       7398      1  0 19:43 ?        00:00:00 redis-server 0.0.0.0:7002 [cluster]
root       7403   3006  0 19:44 pts/0    00:00:00 grep --color=auto redis

检查端口监听状态：

[root@ c7-41 7002]# netstat -tunlp |grep redis
tcp        0      0 0.0.0.0:7000            0.0.0.0:*               LISTEN      7388/redis-server 0
tcp        0      0 0.0.0.0:7001            0.0.0.0:*               LISTEN      7393/redis-server 0
tcp        0      0 0.0.0.0:7002            0.0.0.0:*               LISTEN      7398/redis-server 0
tcp        0      0 0.0.0.0:17000           0.0.0.0:*               LISTEN      7388/redis-server 0
tcp        0      0 0.0.0.0:17001           0.0.0.0:*               LISTEN      7393/redis-server 0
tcp        0      0 0.0.0.0:17002           0.0.0.0:*               LISTEN      7398/redis-server 0
tcp        0      0 0.0.0.0:6379            0.0.0.0:*               LISTEN      7359/redis-server 0

多实例启动后，将7000，7001，7002加入集群
虚拟机二先不用这么操作

[root@ c7-41 7002]# redis-cli --cluster create 0.0.0.0:7000 0.0.0.0:7001 0.0.0.0:7002
>>> Performing hash slots allocation on 3 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
M: ea92d86c515ae7f87280728f53cf303091d7dba9 0.0.0.0:7000
   slots:[0-5460] (5461 slots) master
M: b8bd0c0a7e0b0c3688e15d785838b6fb7c1bbd73 0.0.0.0:7001
   slots:[5461-10922] (5462 slots) master
M: 35cd18904576ddaea65929acac4fa81e89636449 0.0.0.0:7002
   slots:[10923-16383] (5461 slots) master
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 0.0.0.0:7000)
M: ea92d86c515ae7f87280728f53cf303091d7dba9 0.0.0.0:7000
   slots:[0-5460] (5461 slots) master
M: 35cd18904576ddaea65929acac4fa81e89636449 127.0.0.1:7002
   slots:[10923-16383] (5461 slots) master
M: b8bd0c0a7e0b0c3688e15d785838b6fb7c1bbd73 127.0.0.1:7001
   slots:[5461-10922] (5462 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

上面末尾两处OK表示集群创建完成，检查集群包含哪些节点：

[root@ c7-41 7002]# redis-cli -c -p 7000
127.0.0.1:7000> cluster nodes
35cd18904576ddaea65929acac4fa81e89636449 127.0.0.1:7002@17002 master - 0 1617277801605 3 connected 10923-16383
ea92d86c515ae7f87280728f53cf303091d7dba9 127.0.0.1:7000@17000 myself,master - 0 1617277800000 1 connected 0-5460
b8bd0c0a7e0b0c3688e15d785838b6fb7c1bbd73 127.0.0.1:7001@17001 master - 0 1617277800586 2 connected 5461-10922
127.0.0.1:7000>

上面可以看到三个节点全为master节点。
添加新的节点：
把7003节点加入集群：

[root@ c7-41 redis-cluster]# redis-cli --cluster add-node 10.0.0.42:7003 0.0.0.0:7000
>>> Adding node 10.0.0.42:7003 to cluster 0.0.0.0:7000
>>> Performing Cluster Check (using node 0.0.0.0:7000)
M: ea92d86c515ae7f87280728f53cf303091d7dba9 0.0.0.0:7000
   slots:[0-5460] (5461 slots) master
M: 35cd18904576ddaea65929acac4fa81e89636449 127.0.0.1:7002
   slots:[10923-16383] (5461 slots) master
M: b8bd0c0a7e0b0c3688e15d785838b6fb7c1bbd73 127.0.0.1:7001
   slots:[5461-10922] (5462 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 10.0.0.42:7003 to make it join the cluster.
[OK] New node added correctly.

添加新的节点作为现有master节点的slave节点：

[root@ c7-41 7000]# redis-cli --cluster add-node 10.0.0.42:7003 10.0.0.41:7000 --cluster-slave
>>> Adding node 10.0.0.42:7003 to cluster 10.0.0.41:7000
>>> Performing Cluster Check (using node 10.0.0.41:7000)
M: ea92d86c515ae7f87280728f53cf303091d7dba9 10.0.0.41:7000
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 1205cb038661fdf67a4107adc2d204108a13e16c 10.0.0.42:7004
   slots: (0 slots) slave
   replicates ea92d86c515ae7f87280728f53cf303091d7dba9
M: 35cd18904576ddaea65929acac4fa81e89636449 127.0.0.1:7002
   slots:[10923-16383] (5461 slots) master
M: b8bd0c0a7e0b0c3688e15d785838b6fb7c1bbd73 127.0.0.1:7001
   slots:[5461-10922] (5462 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
Automatically selected master 127.0.0.1:7002
>>> Send CLUSTER MEET to node 10.0.0.42:7003 to make it join the cluster.
Waiting for the cluster to join

>>> Configure node as replica of 127.0.0.1:7002.
[OK] New node added correctly.

Configure node as replica of 0.0.0.0（如果这个ip报错，可以使用本机ip）:7000.表示7003节点已经作为7000的slave节点添加进来了。（作者还加了7004节点）
再次查看集群信息：
salve后面的ID是master节点的ID，表示是哪个master节点的slave节点：

[root@ c7-41 redis-cluster]# redis-cli -c -p 7000
127.0.0.1:7000> cluster nodes
b8bd0c0a7e0b0c3688e15d785838b6fb7c1bbd73 127.0.0.1:7001@17001 master - 0 1617327605907 2 connected 5461-10922
35cd18904576ddaea65929acac4fa81e89636449 127.0.0.1:7002@17002 master - 0 1617327604000 3 connected 10923-16383
1205cb038661fdf67a4107adc2d204108a13e16c 10.0.0.42:7004@17004 slave ea92d86c515ae7f87280728f53cf303091d7dba9 0 1617327603000 1 connected
ea92d86c515ae7f87280728f53cf303091d7dba9 10.0.0.41:7000@17000 myself,master - 0 1617327603000 1 connected 0-5460
f9d7effe2d057bd385fa0a0067aea75cfa62e14c 10.0.0.42:7003@17003 slave 35cd18904576ddaea65929acac4fa81e89636449 0 1617327604891 3 connected
127.0.0.1:7000>

检查节点健康情况：

[root@ c7-41 redis-cluster]# redis-cli -c -p 7000
127.0.0.1:7000> info
# Server
redis_version:5.0.4
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:b3caf183223054ed
redis_mode:cluster
os:Linux 3.10.0-1062.el7.x86_64 x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:4.8.5
process_id:7388
run_id:ac7f832c22215d2f4028b76e341425f77700534f
tcp_port:7000
uptime_in_seconds:50488
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:6715118
executable:/usr/local/redis-5.0.4/redis-cluster/7000/redis-server ##启动路径
config_file:/usr/local/redis-5.0.4/redis-cluster/7000/redis.conf	##配置文件路径

# Clients
connected_clients:1
client_recent_max_input_buffer:2
client_recent_max_output_buffer:0
blocked_clients:0

# Memory
used_memory:2635632
used_memory_human:2.51M
used_memory_rss:18698240
used_memory_rss_human:17.83M
used_memory_peak:5638552
used_memory_peak_human:5.38M
used_memory_peak_perc:46.74%
used_memory_overhead:2565736
used_memory_startup:1450544
used_memory_dataset:69896
used_memory_dataset_perc:5.90%
allocator_allocated:3054816
allocator_active:3346432
allocator_resident:15015936
total_system_memory:1019645952
total_system_memory_human:972.41M
used_memory_lua:37888
used_memory_lua_human:37.00K
used_memory_scripts:0
used_memory_scripts_human:0B
number_of_cached_scripts:0
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.10
allocator_frag_bytes:291616
allocator_rss_ratio:4.49
allocator_rss_bytes:11669504
rss_overhead_ratio:1.25
rss_overhead_bytes:3682304
mem_fragmentation_ratio:7.21
mem_fragmentation_bytes:16104864
mem_not_counted_for_evict:0
mem_replication_backlog:1048576
mem_clients_slaves:16922
mem_clients_normal:49694
mem_aof_buffer:0
mem_allocator:jemalloc-5.1.0
active_defrag_running:0
lazyfree_pending_objects:0

# Persistence
loading:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1617327500
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:0
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:4390912
aof_enabled:1	#当前aof是开启的，如需关闭，需在配置文件中把appendonly yes改为appendonly no
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
aof_current_size:0
aof_base_size:0
aof_pending_rewrite:0
aof_buffer_length:0
aof_rewrite_buffer_length:0
aof_pending_bio_fsync:0
aof_delayed_fsync:0

# Stats
total_connections_received:24
total_commands_processed:3272
instantaneous_ops_per_sec:1
total_net_input_bytes:147578
total_net_output_bytes:1120837
instantaneous_input_kbps:0.04
instantaneous_output_kbps:0.00
rejected_connections:0
sync_full:1
sync_partial_ok:0
sync_partial_err:1
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
evicted_keys:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:2725
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0

# Replication
role:master
connected_slaves:1
slave0:ip=10.0.0.42,port=7004,state=online,offset=476,lag=0
master_replid:6ac36648055903733cbb7252ca497483277976bc
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:476
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:476

# CPU
used_cpu_sys:4.732625
used_cpu_user:5.270087
used_cpu_sys_children:0.001653
used_cpu_user_children:0.000826

# Cluster
cluster_enabled:1

# Keyspace
127.0.0.1:7000>

此时检查6379节点slots插槽是否完全覆盖：

[root@ c7-41 redis-cluster]# redis-cli --cluster check 0.0.0.0:7000
0.0.0.0:7000 (ea92d86c...) -> 0 keys | 5461 slots | 1 slaves.
127.0.0.1:7001 (b8bd0c0a...) -> 0 keys | 5462 slots | 0 slaves.
127.0.0.1:7002 (35cd1890...) -> 0 keys | 5461 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 0.0.0.0:7000)
M: ea92d86c515ae7f87280728f53cf303091d7dba9 0.0.0.0:7000
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: b8bd0c0a7e0b0c3688e15d785838b6fb7c1bbd73 127.0.0.1:7001
   slots:[5461-10922] (5462 slots) master
M: 35cd18904576ddaea65929acac4fa81e89636449 127.0.0.1:7002
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 1205cb038661fdf67a4107adc2d204108a13e16c 10.0.0.42:7004
   slots: (0 slots) slave
   replicates ea92d86c515ae7f87280728f53cf303091d7dba9
S: f9d7effe2d057bd385fa0a0067aea75cfa62e14c 10.0.0.42:7003
   slots: (0 slots) slave
   replicates 35cd18904576ddaea65929acac4fa81e89636449
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

处理未完全覆盖的情况：

[root@ c7-41 redis-cluster]# redis-cli --cluster fix 0.0.0.0:7000

redis基本使用 config命令

[root@ c7-41 redis-cluster]# redis-cli -c -p 7000
127.0.0.1:7000> config get appendonly	#config get 查看指定某个配置项
1) "appendonly"
2) "yes"
127.0.0.1:7000> config set appendonly no	#set 是将某个配置项修改为新的内容
OK
127.0.0.1:7000> config get appendonly	#config表示查看所有的配置项
1) "appendonly"
2) "no"
127.0.0.1:7000>

数据类型：
string字符串，是redis最基本的数据类型，最大支持512M，每个key对应一个value string是二进制的，可以包含任何数据，包括图像或序列化的对象

[root@ c7-41 redis-cluster]# redis-cli -c -p 7000
127.0.0.1:7000> set name ycf
-> Redirected to slot [5798] located at 127.0.0.1:7001
OK
127.0.0.1:7001> get name
"ycf"
127.0.0.1:7001>

hash是一个键值的集合，包含了key和value;
添加hash之前先把上面创建的ycf删掉，不然会报错：(error) WRONGTYPE Operation against a key holding the wrong kind of value

127.0.0.1:7001> del name
(integer) 1
127.0.0.1:7001> hmset myhash field1 'hello' filed2 'world'
OK
127.0.0.1:7001> hget myhash field1
"hello"
127.0.0.1:7001> hget myhash filed2
"world"

实例中我们使用了 redis hmset, hget 命令，hmset 设置了两个 field=>value 对,hget 获取对应 field 对应的 value。
每个 hash 可以存储 232 -1 键值对（40多亿）

备份与恢复：
redis备份使用save命令，备份完成后会在文件目录中出现dump.rdb文件，备份文件存放位置可使用CONFIG get dir 来查看。创建 redis 备份文件也可以使用命令 BGSAVE，该命令在后台执行。

[root@ c7-41 redis-cluster]# redis-cli -c -p 7000
127.0.0.1:7000> save
OK
127.0.0.1:7000> CONFIG get dir
1) "dir"
2) "/usr/local/redis-5.0.4/redis-cluster/7000"	#此为redis集群安装位置，也是数据库文件存放路径
127.0.0.1:7000> BGSAVE
Background saving started
127.0.0.1:7000>
[root@ c7-41 redis-cluster]# pwd
/usr/local/redis-5.0.4/redis-cluster
[root@ c7-41 redis-cluster]# ll
total 0
drwxr-xr-x 2 root root 85 Apr  2 10:08 7000
drwxr-xr-x 2 root root 69 Apr  1 21:11 7001
drwxr-xr-x 2 root root 85 Apr  2 09:38 7002
[root@ c7-41 redis-cluster]# cd 7000
[root@ c7-41 7000]# ll
total 16
-rw-r--r-- 1 root root  92 Apr  2 09:53 appendonly.aof
-rw-r--r-- 1 root root 176 Apr  2 10:08 dump.rdb
-rw-r--r-- 1 root root 644 Apr  2 09:38 nodes_7000.conf
-rw-r--r-- 1 root root 207 Apr  1 19:37 redis.conf

如果需要恢复数据，只需将备份文件 (dump.rdb) 移动到 redis 安装目录并启动服务即可。

redis配置密码：

[root@ c7-41 7000]# redis-cli -c -p 7000
127.0.0.1:7000> config get requirepass
1) "requirepass"	#查看是否设置密码
2) ""	#当前为空，说明未设置密码
127.0.0.1:7000> config set requirepass '123456'	#此处设置密码
OK
127.0.0.1:7000> config get requirepass
(error) NOAUTH Authentication required.	#此处提示已经设置了密码，需要输入密码
127.0.0.1:7000> auth '123456'	#输入密码的方式为 auth ‘密码’
OK
127.0.0.1:7000> config get requirepass
1) "requirepass"
2) "123456"
127.0.0.1:7000>

redis性能测试：
redis性能测试使用的命令格式为：
redis-benchmark [option] [option value]
该命令在服务器模式下执行，不是在redis上执行：

[root@ c7-41 7000]# redis-benchmark -n 10000 -q
PING_INLINE: 72463.77 requests per second
PING_BULK: 72992.70 requests per second
SET: 43478.26 requests per second
GET: 68027.21 requests per second
INCR: 70921.98 requests per second
LPUSH: 68027.21 requests per second
RPUSH: 68027.21 requests per second
LPOP: 68965.52 requests per second
RPOP: 67567.57 requests per second
SADD: 69444.45 requests per second
HSET: 67567.57 requests per second
SPOP: 72463.77 requests per second
LPUSH (needed to benchmark LRANGE): 65789.48 requests per second
LRANGE_100 (first 100 elements): 35087.72 requests per second
LRANGE_300 (first 300 elements): 15527.95 requests per second
LRANGE_500 (first 450 elements): 11402.51 requests per second
LRANGE_600 (first 600 elements): 9124.09 requests per second
MSET (10 keys): 60240.96 requests per second