互联网框架day10(redis-cluster)

快乐咸鱼Y

已于 2023-06-10 03:13:48 修改

阅读量141

点赞数

分类专栏：互联网框架文章标签： redis 缓存数据库

于 2021-04-28 13:51:35 首次发布

本文链接：https://blog.csdn.net/yygyj/article/details/116231497

版权

互联网框架专栏收录该内容

22 篇文章 1 订阅

订阅专栏

[如有侵权联系删除]
在这里插入图片描述

3.首次接触槽道的概念

在这里插入图片描述

在这里插入图片描述
拷贝一份文件进行编辑

在这里插入图片描述

干掉无用进程
在这里插入图片描述

[root@localhost02 bin]# mkdir 8000 8001 8002 8003 8004 8005
[root@localhost02 bin]# ls
6379redis.log  8000  dump6379.rdb  dump.rdb        redis6383.conf    redis-cluster.conf
6380redis.log  8001  dump6380.rdb  redis2.conf     redis6384.conf    redis.conf
6381redis.log  8002  dump6381.rdb  redis6379.conf  redis-benchmark   redis.log
6382redis.log  8003  dump6382.rdb  redis6380.conf  redis-check-aof   redis-sentinel
6383redis.log  8004  dump6383.rdb  redis6381.conf  redis-check-dump  redis-server
6384redis.log  8005  dump6384.rdb  redis6382.conf  redis-cli         set
[root@localhost02 bin]# cp redis-cluster.conf 8000
[root@localhost02 bin]# cp redis-cluster.conf 8001
[root@localhost02 bin]# cp redis-cluster.conf 8002
[root@localhost02 bin]# cp redis-cluster.conf 8003
[root@localhost02 bin]# cp redis-cluster.conf 8004
[root@localhost02 bin]# cp redis-cluster.conf 8005

递归查看模板文件

[root@localhost02 bin]# ls -R 800*
8000:
redis-cluster.conf

8001:
redis-cluster.conf

8002:
redis-cluster.conf

8003:
redis-cluster.conf

8004:
redis-cluster.conf

8005:
redis-cluster.conf

在这里插入图片描述

[root@localhost02 bin]# ./redis-server 8000/redis-cluster.conf
[root@localhost02 bin]# ./redis-server 8001/redis-cluster.conf
[root@localhost02 bin]# ./redis-server 8002/redis-cluster.conf
[root@localhost02 bin]# ./redis-server 8003/redis-cluster.conf
[root@localhost02 bin]# ./redis-server 8004/redis-cluster.conf
[root@localhost02 bin]# ./redis-server 8005/redis-cluster.conf

[root@localhost02 bin]# ps -ef|grep redis
root     21443     1  0 16:41 ?        00:00:00 ./redis-server *:8000 [cluster]       
root     21447     1  0 16:41 ?        00:00:00 ./redis-server *:8001 [cluster]       
root     21452     1  0 16:41 ?        00:00:00 ./redis-server *:8002 [cluster]       
root     21456     1  0 16:41 ?        00:00:00 ./redis-server *:8003 [cluster]       
root     21460     1  0 16:41 ?        00:00:00 ./redis-server *:8004 [cluster]       
root     21464     1  0 16:42 ?        00:00:00 ./redis-server *:8005 [cluster]       
root     21470  1496  0 16:42 pts/2    00:00:00 grep redis

[root@localhost02 bin]# ./redis-cli -p 8000
127.0.0.1:8000> get keys *
(error) ERR wrong number of arguments for 'get' command

在这里插入图片描述
无法新建。hash槽没有分配

[root@localhost02 bin]# ./redis-cli -c  -p 8000
127.0.0.1:8000> info replication
# Replication
role:master
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
127.0.0.1:8000> set name haha
(error) CLUSTERDOWN The cluster is down

查看信息【证明状态不可用】

127.0.0.1:8000> cluster info
cluster_state:fail
cluster_slots_assigned:0
cluster_slots_ok:0
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:1
cluster_size:0
cluster_current_epoch:0
cluster_my_epoch:0
cluster_stats_messages_sent:0
cluster_stats_messages_received:0

在这里插入图片描述

在这里插入图片描述
注意：

./redis-trib.rb create --replicas 1 172.16.220.151:6379 172.16.220.152:6379 172.16.220.153:6379 172.16.220.154:6379
/usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require': no such file to load -- redis (LoadError)
	from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
	from ./redis-trib.rb:25

解决方案


gem install redis

如果已经执行还不行，重启虚拟机

启动redis-cluster

[root@localhost02 src]# ./redis-trib.rb create 192.168.253.129:8000 192.168.253.129:8001 192.168.253.129:8002

启动一个节点

[root@localhost02 src]# ./redis-cli -c -p 8000

信息展示

127.0.0.1:8000> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:3
cluster_size:3
cluster_current_epoch:3
cluster_my_epoch:1
cluster_stats_messages_sent:15362
cluster_stats_messages_received:15362

两两互联

127.0.0.1:8000> cluster nodes
40e49529e3f857dc729737cc98641d1f337b1473 192.168.253.129:8001 master - 0 1620562166046 2 connected 5461-10922
9c8fe9b37bc3e9cd90ac7f307824ec9b7e5abb82 192.168.253.129:8002 master - 0 1620562165039 3 connected 10923-16383
086b638aaf64d8967c9ff5a385bd10e96b1f1cd5 192.168.253.129:8000 myself,master - 0 0 1 connected 0-5460

在这里插入图片描述

8003相当于原来的哨兵，起监控作用，没有存储作用【主节点】

[root@localhost02 src]# ./redis-trib.rb add-node 192.168.253.129:8003 192.168.253.129:8000

在这里插入图片描述

[root@localhost02 src]# ./redis-trib.rb add-node --slave --master-id 40e49529e3f857dc729737cc98641d1f337b1473 192.168.253.129:8004 192.168.253.129:8001

在这里插入图片描述
当把8001恢复，8001变为8004的从节点

在这里插入图片描述

8003为空槽道【要利用起来（迁移）】

在这里插入图片描述

这里的8000是主节点
在这里插入图片描述

删除之后再添加【有问题】

[root@localhost02 src]# ./redis-trib.rb add-node 192.168.253.129:8001 192.168.253.129:8000
>>> Adding node 192.168.253.129:8001 to cluster 192.168.253.129:8000
>>> Performing Cluster Check (using node 192.168.253.129:8000)
M: 086b638aaf64d8967c9ff5a385bd10e96b1f1cd5 192.168.253.129:8000
   slots:66-5460 (5395 slots) master
   0 additional replica(s)
M: f5638cd1021728d1c1791dc4ef75a275433b3e65 192.168.253.129:8003
   slots:0-65,5461-5527,10923-10988 (199 slots) master
   0 additional replica(s)
M: 9c8fe9b37bc3e9cd90ac7f307824ec9b7e5abb82 192.168.253.129:8002
   slots:10989-16383 (5395 slots) master
   0 additional replica(s)
M: a388daf5e10f93f85a453e90ede1e3225b84a7fe 192.168.253.129:8004
   slots:5528-10922 (5395 slots) master
   0 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[ERR] Node 192.168.253.129:8001 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.

在这里插入图片描述

删除方法【删除aof rdb node】

[root@localhost02 src]# ./redis-cli -c -p 8001
127.0.0.1:8001> shutdown
not connected> quit
[root@localhost02 redis]# cd bin
[root@localhost02 bin]# ls
6379redis.log  8003                dump6379.rdb     nodes-8003.conf  redis-check-dump
6380redis.log  8003redis.log       dump6380.rdb     nodes-8004.conf  redis-cli
6381redis.log  8004                dump6381.rdb     nodes-8005.conf  redis-cluster.conf
6382redis.log  8004redis.log       dump6382.rdb     redis2.conf      redis.conf
6383redis.log  8005                dump6383.rdb     redis6379.conf   redis.log
6384redis.log  8005redis.log       dump6384.rdb     redis6380.conf   redis-sentinel
8000           appendonly8000.aof  dump8001.rdb     redis6381.conf   redis-server
8000redis.log  appendonly8001.aof  dump8004.rdb     redis6382.conf   set
8001           appendonly8002.aof  dump.rdb         redis6383.conf
8001redis.log  appendonly8003.aof  nodes-8000.conf  redis6384.conf
8002           appendonly8004.aof  nodes-8001.conf  redis-benchmark
8002redis.log  appendonly8005.aof  nodes-8002.conf  redis-check-aof
[root@localhost02 bin]# rm -f dump8001.rdb
[root@localhost02 bin]# rm -f appendonly8001.aof
[root@localhost02 bin]# rm -f nodes-8001.conf

在这里插入图片描述

[root@hadoop01 src]# ./redis-cli -c -p 8000 shutdown
[root@hadoop01 src]# ./redis-cli -c -p 8001 shutdown
[root@hadoop01 src]# ./redis-cli -c -p 8002 shutdown
[root@hadoop01 src]# ./redis-cli -c -p 8003 shutdown
[root@hadoop01 src]# ./redis-cli -c -p 8004 shutdown
[root@hadoop01 src]# ./redis-cli -c -p 8005 shutdown

[root@hadoop01 bin]# rm -f appendonly800*
[root@hadoop01 bin]# rm -f dump800*
[root@hadoop01 bin]# rm -f nodes-800*

在这里插入图片描述

[root@hadoop01 bin]# ./redis-server 8000/redis-cluster.conf &

[root@hadoop01 bin]# ./redis-server 8001/redis-cluster.conf &

[root@hadoop01 bin]# ./redis-server 8002/redis-cluster.conf &

[root@hadoop01 bin]# ./redis-server 8003/redis-cluster.conf &

[root@hadoop01 bin]# ./redis-server 8004/redis-cluster.conf &

[root@hadoop01 bin]# ./redis-server 8005/redis-cluster.conf &

在这里插入图片描述

1.4建（ruby）
./redis-trib.rb create --replicas 1 192.168.253.129:8000 192.168.253.129:8001 192.168.253.129:8002 192.168.253.129:8003 192.168.253.129:8004 192.168.253.129:8005

效果（自动分配三主三从）

[root@hadoop01 src]# ./redis-cli -c -p 8000
127.0.0.1:8000> CLUSTER NODES
d27b817dd763233e919954e3e21eb33adb350fb6 192.168.253.129:8002 master - 0 1686203390513 3 connected 10923-16383
c51cdde14d5ff2408c9258b984fe4d94abc1bdac 192.168.253.129:8004 slave 0961690adebe271e17fe9baf0e79a3eeab73af3e 0 1686203391517 5 connected
c601bbd3bd70f5daa5edccc46b413c29d0fc27e6 192.168.253.129:8005 slave d27b817dd763233e919954e3e21eb33adb350fb6 0 1686203387495 6 connected
0961690adebe271e17fe9baf0e79a3eeab73af3e 192.168.253.129:8001 master - 0 1686203386488 2 connected 5461-10922
ae87c7d05c66445d38ba5f7339371019a876a1bb 192.168.253.129:8003 slave c477823eba841a0964e3aea8b6404e4291884f7a 0 1686203389507 4 connected
c477823eba841a0964e3aea8b6404e4291884f7a 192.168.253.129:8000 myself,master - 0 0 1 connected 0-5460

1.理解槽道原理，回答两个问题【难点（重要面试未必问）】
如何判断计算出来的槽道号是否归属于当前的节点？
不归属如何寻找到正确的节点信息返回？
在这里插入图片描述
2.槽道原理
2.1集群启动时创建的内存数据
启动单个节点（相互之间谁也不与其它人通信）

所以集群创建之后，登录任何一个节点，都可以通过cluster node查看整个集群所有的节点信息

2.2槽道的结构
在这里插入图片描述
可以利用槽道号从二进制获取下标对应的二进制的值，1/0，如果是1，表示当前槽道归属true，0表示不归属false

在这里插入图片描述

在这里插入图片描述
没有二进制直接使用数组可行？

redis服务端接收set命令处理

停删启建（重建集群）

启动节点【如果有残留数据（停删启建）】
在这里插入图片描述

[root@localhost02 src]# ./redis-cli -c -p 8000 -h 192.168.253.129
192.168.253.129:8000> cluster nodes
769453ed31c5cb15e7e9e4be1b98adea3d1c0305 :8000 myself,master - 0 0 0 connected

在这里插入图片描述
两两互联

192.168.253.129:8000> cluster meet 192.168.253.129 8000
OK
192.168.253.129:8000> cluster meet 192.168.253.129 8001
OK
192.168.253.129:8000> cluster meet 192.168.253.129 8002
192.168.253.129:8000> cluster meet 192.168.253.129 8003
OK
192.168.253.129:8000> cluster meet 192.168.253.129 8004
OK
192.168.253.129:8000> cluster meet 192.168.253.129 8005
OK

编写脚本,执行脚本

[root@localhost02 src]# vim slot.sh
for slot in {0..5460};do ./redis-cli -c -p 8000 cluster addslots $slot;done;
for slot in {5461..10922};do ./redis-cli -c -p 8001 cluster addslots $slot;done;
for slot in {10923..16383};do ./redis-cli -c -p 8002 cluster addslots $slot;done;
[root@localhost02 src]# sh slot.sh

槽道已经分配

[root@localhost02 src]# ./redis-cli -c -p 8000
127.0.0.1:8000> cluster nodes
bbbeed5623665457161b583d0f95208d1926970e 192.168.253.129:8002 master - 0 1620708623663 2 connected 10923-16383
769453ed31c5cb15e7e9e4be1b98adea3d1c0305 192.168.253.129:8000 myself,master - 0 0 1 connected 0-5460
64fc75ebd40d94dd6fdbd6c11f98414b9a3713c3 192.168.253.129:8003 master - 0 1620708620136 3 connected
85780d7c78e3a4d88bd76d81a25b2979fea4ae12 192.168.253.129:8004 master - 0 1620708620642 4 connected
50e4ac99fd7a163eb23efc9357109fe84ffbad4f 192.168.253.129:8005 master - 0 1620708622656 5 connected
cf08a80cadb5229a76736bd282e7f667b388dcab 192.168.253.129:8001 master - 0 1620708624671 0 connected 5461-10922

在这里插入图片描述

[root@localhost02 src]# ./redis-cli -c -p 8003
127.0.0.1:8003> cluster replicate cf08a80cadb5229a76736bd282e7f667b388dcab
OK

总结：redis的分布式集群【可以通过ruby脚本也可以不用ruby脚本，用上述的sh slot.sh】

在这里插入图片描述

在这里插入图片描述

非空槽道的迁移
槽道的迁移和数据的迁移，还得告诉当事人。

编写脚本【停删启创】

[root@localhost02 src]# vim recreate.sh
for port in {8000..8005};do ./redis-cli -c -p $port shutdown;done;
rm -rf /usr/local/redis/bin/dump800*;
rm -rf /usr/local/redis/bin/appendonly800*;
rm -rf /usr/local/redis/bin/nodes-800*;
for port in {8000..8005};do ./redis-server /usr/local/redis/bin/$port/redis-cluster.conf;done;
echo yes|./redis-trib.rb create --replicas 1 192.168.253.129:8000 192.168.253.129:8001 192.168.253.129:8002 192.168.253.129:8003 192.168.253.129:8004 192.168.253.129:8005;
[root@localhost02 src]# sh recreate.sh

[root@localhost02 src]# sh recreate.sh
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.253.129:8000
192.168.253.129:8001
192.168.253.129:8002
Adding replica 192.168.253.129:8003 to 192.168.253.129:8000
Adding replica 192.168.253.129:8004 to 192.168.253.129:8001
Adding replica 192.168.253.129:8005 to 192.168.253.129:8002
M: 5676688e6330799acb7d13cf05caf2a314dbe1dd 192.168.253.129:8000
   slots:0-5460 (5461 slots) master
M: 4a316ae2c82ae3004c5e97153421c38a9104219b 192.168.253.129:8001
   slots:5461-10922 (5462 slots) master
M: 2ad79b748fda169b53333bcc87b4a2e283950640 192.168.253.129:8002
   slots:10923-16383 (5461 slots) master
S: 412d6506990ea5b413c0ec16e62d49a9f2e8b09b 192.168.253.129:8003
   replicates 5676688e6330799acb7d13cf05caf2a314dbe1dd
S: 14e3b54064a9f1c4bfd545db59818c6d8a8c36f3 192.168.253.129:8004
   replicates 4a316ae2c82ae3004c5e97153421c38a9104219b
S: c1dc6a527b0b8b8b3295f141028e9a2870c6b719 192.168.253.129:8005
   replicates 2ad79b748fda169b53333bcc87b4a2e283950640
Can I set the above configuration? (type 'yes' to accept): >>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join........
>>> Performing Cluster Check (using node 192.168.253.129:8000)
M: 5676688e6330799acb7d13cf05caf2a314dbe1dd 192.168.253.129:8000
   slots:0-5460 (5461 slots) master
M: 4a316ae2c82ae3004c5e97153421c38a9104219b 192.168.253.129:8001
   slots:5461-10922 (5462 slots) master
M: 2ad79b748fda169b53333bcc87b4a2e283950640 192.168.253.129:8002
   slots:10923-16383 (5461 slots) master
M: 412d6506990ea5b413c0ec16e62d49a9f2e8b09b 192.168.253.129:8003
   slots: (0 slots) master
   replicates 5676688e6330799acb7d13cf05caf2a314dbe1dd
M: 14e3b54064a9f1c4bfd545db59818c6d8a8c36f3 192.168.253.129:8004
   slots: (0 slots) master
   replicates 4a316ae2c82ae3004c5e97153421c38a9104219b
M: c1dc6a527b0b8b8b3295f141028e9a2870c6b719 192.168.253.129:8005
   slots: (0 slots) master
   replicates 2ad79b748fda169b53333bcc87b4a2e283950640
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@localhost02 src]# ./redis-cli -c -p 8000
127.0.0.1:8000> cluster nodes
c1dc6a527b0b8b8b3295f141028e9a2870c6b719 192.168.253.129:8005 slave 2ad79b748fda169b53333bcc87b4a2e283950640 0 1620712263287 6 connected
2ad79b748fda169b53333bcc87b4a2e283950640 192.168.253.129:8002 master - 0 1620712264795 3 connected 10923-16383
4a316ae2c82ae3004c5e97153421c38a9104219b 192.168.253.129:8001 master - 0 1620712261775 2 connected 5461-10922
5676688e6330799acb7d13cf05caf2a314dbe1dd 192.168.253.129:8000 myself,master - 0 0 1 connected 0-5460
412d6506990ea5b413c0ec16e62d49a9f2e8b09b 192.168.253.129:8003 slave 5676688e6330799acb7d13cf05caf2a314dbe1dd 0 1620712262784 4 connected
14e3b54064a9f1c4bfd545db59818c6d8a8c36f3 192.168.253.129:8004 slave 4a316ae2c82ae3004c5e97153421c38a9104219b 0 1620712263790 5 connected

JedisCluster连接集群
1.jedisCluster的使用
在这里插入图片描述
1.1测试代码
实现三主各一从的redis-cluster结构，实现操作代码逻辑

@Test
	public void jedisCluster(){
		//手机节点信息，至少提供一个(两两互联)[如果8000没有开启则不能连接，不是高可用]
		Set<HostAndPort> clusterSet=new HashSet<HostAndPort>();
		clusterSet.add(new HostAndPort("192.168.253.129", 8000));
		//cluster对象底层使用连接池，使用config配置连接池
		GenericObjectPoolConfig config=new GenericObjectPoolConfig();
		config.setMaxTotal(200);
		//创建JedisCluster
		JedisCluster cluster=new JedisCluster(clusterSet,config);
		cluster.set("name", "啊啊啊");//直接在8001，没有重定向
		System.out.println(cluster.get("name"));
		
	}