本站以分享各种运维经验和运维所需要的技能为主
《python零基础入门》:python零基础入门学习
《python运维脚本》: python运维脚本实践
《shell》:shell学习
《terraform》持续更新中:terraform_Aws学习零基础入门到最佳实战
《k8》从问题中去学习k8s
《docker学习》暂未更新
《ceph学习》ceph日常问题解决分享
《日志收集》ELK+各种中间件
《运维日常》运维日常
《linux》运维面试100问
一、Redis Cluster 分布式集群
1.什么是Redis Cluster
1.Redis集群是一个可以在多个Redis节点之间进行数据共享的设施(installation) 2.Redis集群不支持那些需要同时处理多个键的Redis命令,因为执行这些命令需要在多个Redis节点之间移动数据,并且在高负载的情况下,这些命令将降低Redis集群的性能,并导致不可预测的行为。(使用ack协议) 3.Redis集群通过分区(partition)来提供一定程度的可用性(availability):即使集群中有一部分节点失效或者无法进行通讯,集群也可以继续处理命令请求。 4.Redis集群有将数据自动切分(split)到多个节点的能力。
2.Redis Cluster的特点
#高性能: 1.在多分片节点中,将16384个槽位,均匀分布到多个分片节点中 2.存数据时,将key做crc16(key),然后和16384进行取模,得出槽位值(0-16384之间) 3.根据计算得出的槽位值,找到相对应的分片节点的主节点,存储到相应槽位上 4.如果客户端当时连接的节点不是将来要存储的分片节点,分片集群会将客户端连接切换至真正存储节点进行数据存储 5.客户端与redis节点直连,不需要中间proxy层.客户端不需要连接集群所有节点,连接集群中任何一个可用节点即可 6.Redis Cluster解决了redis资源利用率的问题 #高可用 7.在搭建集群时,会为每一个分片的主节点,对应一个从节点,实现slaveof功能,同时当主节点down,实现类似于sentinel的自动failover的功能。
3.槽的概念
1.在集群中,会把所有节点分为16384个槽位 2.槽位的序号是 0 - 16383,序号不重要,数量才重要 3.每一个槽位分配到数据的概率是一样
4.redis故障转移
1.在集群里面,节点会对其他节点进行下线检测。 2.当一个主节点下线时,集群里面的其他主节点负责对下线主节点进行故障移。 3.换句话说,集群的节点集成了下线检测和故障转移等类似 Sentinel 的功能。
二、redis集群搭建(方法一)
须知
# 如下只是集群关联的一种方式,还有一种方式需要安装相应工具,但那种方式比较简单,但会自动进行主从,缺点也存在,会自动进行主从,如果不是想要的主从状态,将要自己手动修改。(详情见方法二)
1.环境准备
节点 | IP | 端口 |
---|---|---|
节点1 | 172.16.1.51 | 6379,6380 |
节点2 | 172.16.1.52 | 6379,6380 |
节点3 | 172.16.1.53 | 6379,6380 |
2.搭建redis
#删除以前的redis数据 [root@db01 ~]# rm -rf /service/redis/* #创建多实例目录 [root@db01 ~]# mkdir /service/redis/{6379,6380} [root@db02 ~]# mkdir /service/redis/{6379,6380} [root@db03 ~]# mkdir /service/redis/{6379,6380} #配置所有redis [root@db01 ~]# vim /service/redis/6379/redis.conf bind 172.16.1.51 127.0.0.1 port 6379 daemonize yes pidfile /service/redis/6379/redis.pid loglevel notice logfile /service/redis/6379/redis.log dbfilename dump.rdb dir /service/redis/6379 cluster-enabled yes # 打开redis集群 cluster-config-file nodes.conf # 集群关联文件地址 cluster-node-timeout 5000 # 节点互联超时阈值 [root@db01 ~]# vim /service/redis/6380/redis.conf bind 172.16.1.51 127.0.0.1 port 6380 daemonize yes pidfile /service/redis/6380/redis.pid loglevel notice logfile /service/redis/6380/redis.log dbfilename dump.rdb dir /service/redis/6380 cluster-enabled yes cluster-config-file nodes.conf cluster-node-timeout 5000
3.启动所有redis
[root@db01 ~]# redis-server /service/redis/6379/redis.conf [root@db01 ~]# redis-server /service/redis/6380/redis.conf [root@db02 ~]# redis-server /service/redis/6379/redis.conf [root@db02 ~]# redis-server /service/redis/6380/redis.conf [root@db03 ~]# redis-server /service/redis/6379/redis.conf [root@db03 ~]# redis-server /service/redis/6380/redis.conf
4.关联所有redis节点
1)登录所有节点
[root@db01 ~]# redis-cli -h 172.16.1.51 -p 6379 [root@db01 ~]# redis-cli -h 172.16.1.51 -p 6380 [root@db02 ~]# redis-cli -h 172.16.1.52 -p 6379 [root@db02 ~]# redis-cli -h 172.16.1.52 -p 6380 [root@db03 ~]# redis-cli -h 172.16.1.53 -p 6379 [root@db03 ~]# redis-cli -h 172.16.1.53 -p 6380
2)查看集群节点
#查看集群节 79 myself,master - 0 0 0 connected
3)关联所有节点
# 将需要加入集群中的主从服务器全部进行关联 172.16.1.51:6379> CLUSTER MEET 172.16.1.51 6380 OK 172.16.1.51:6379> CLUSTER MEET 172.16.1.52 6379 OK 172.16.1.51:6379> CLUSTER MEET 172.16.1.52 6380 OK 172.16.1.51:6379> CLUSTER MEET 172.16.1.53 6379 OK 172.16.1.51:6379> CLUSTER MEET 172.16.1.53 6380 OK #查看集群状态,所有节点 172.16.1.51:6379> CLUSTER NODES aee9f4e6e09a452fd44bca7639be442b5138f141 172.16.1.52:6380 master - 0 1596687131655 4 connected 777412c8d6554e3390e1083bf1f55002be08cf62 172.16.1.51:6380 master - 0 1596687131352 2 connected ef18ab5bab6d8bc06917a0cf2dc9bffa8b431087 172.16.1.52:6379 master - 0 1596687132362 3 connected f2747c92813ea06b25c3e9c8d5232b46b3cf9d3d 172.16.1.53:6379 master - 0 1596687131856 0 connected 25f735f08ac62b2f758c1e2c21e178cc46279087 172.16.1.53:6380 master - 0 1596687131251 5 connected 28faba09f4c0ec8cdb90d92e09636796427b7143 172.16.1.51:6379 myself,master - 0 0 1 connected
5.分配槽位
#查看集群状态 172.16.1.51:6379> CLUSTER INFO cluster_state:fail cluster_slots_assigned:0 cluster_slots_ok:0 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:0 cluster_current_epoch:5 cluster_my_epoch:1 cluster_stats_messages_sent:1168 cluster_stats_messages_received:1168 #槽位规划 db01: 5462 个槽位 (0-5461) db02: 5461 个槽位 (5462-10922) db03: 5461 个槽位 (10923-16383) #分配槽位 [root@db01 ~]# redis-cli -p 6379 -h 172.16.1.51 CLUSTER ADDSLOTS {0..5461} OK [root@db02 ~]# redis-cli -p 6379 -h 172.16.1.52 CLUSTER ADDSLOTS {5462..10922} OK [root@db02 ~]# redis-cli -p 6379 -h 172.16.1.53 CLUSTER ADDSLOTS {10923..16383}
6.插入数据测试集群
#插入一条数据 172.16.1.51:6379> set k1 v1 (error) MOVED 12706 172.16.1.53:6379 #报错,该key的值只能插入到12706这个槽位 [root@db03 ~]# redis-cli -h 172.16.1.53 172.16.1.53:6379> set k1 v1 OK #ASK协议,自动切换将数据添加到指定槽位(加一个-c参数即可) [root@db03 ~]# redis-cli -h 172.16.1.53 172.16.1.53:6379> set k2 v2 (error) MOVED 449 172.16.1.51:6379 172.16.1.53:6379> quit [root@db03 ~]# redis-cli -c -h 172.16.1.53 172.16.1.53:6379> set k2 v2 -> Redirected to slot [449] located at 172.16.1.51:6379 OK #脚本插入数据测试 [root@db03 ~]# vim data.sh #!/bin/bash for i in {1..1000};do redis-cli -c -p 6379 -h 172.16.1.51 set k${i} v${i} done #查看数据分配 172.16.1.51:6379> DBSIZE (integer) 341 172.16.1.52:6379> DBSIZE (integer) 332 172.16.1.53:6379> DBSIZE (integer) 327
7.添加副本节点
1)查看节点
172.16.1.51:6379> CLUSTER NODES 5a7f0cf95e1850b5b5ae81d873c4c76fd366d604 172.16.1.51:6380 master - 0 1596763193422 4 connected 5eb9e5356534ff4acda736d13f0dc9fc3d40049b 172.16.1.52:6379 master - 0 1596763192412 5 connected 5462-10922 50878ef6a4d8141c8dbca3e2bf7c84ed48a73ee2 172.16.1.53:6380 master - 0 1596763192512 3 connected acc3a4d0e6e43fc74630c1f0714865fdcbdaf677 172.16.1.53:6379 master - 0 1596763191908 0 connected 10923-16383 2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 172.16.1.51:6379 myself,master - 0 0 1 connected 0-5461 381b54584572e8013becdae2eeaff48bf6eb5450 172.16.1.52:6380 master - 0 1596763193925 2 connected
2)配置主从
#db01的6380做db02的6379的从库 172.16.1.51:6380> CLUSTER REPLICATE 5eb9e5356534ff4acda736d13f0dc9fc3d40049b OK #db02的6380做db03的6379的从库 172.16.1.52:6380> CLUSTER REPLICATE acc3a4d0e6e43fc74630c1f0714865fdcbdaf677 OK #db03的6380做db01的6379的从库 172.16.1.53:6380> CLUSTER REPLICATE 2325be6f1f9c1c9f57d5a033fc05e0d798ea823a OK
3)再次查看节点信息
172.16.1.51:6379> CLUSTER NODES 5a7f0cf95e1850b5b5ae81d873c4c76fd366d604 172.16.1.51:6380 slave 5eb9e5356534ff4acda736d13f0dc9fc3d40049b 0 1596763362696 5 connected 5eb9e5356534ff4acda736d13f0dc9fc3d40049b 172.16.1.52:6379 master - 0 1596763363202 5 connected 5462-10922 50878ef6a4d8141c8dbca3e2bf7c84ed48a73ee2 172.16.1.53:6380 slave 2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 0 1596763362192 3 connected acc3a4d0e6e43fc74630c1f0714865fdcbdaf677 172.16.1.53:6379 master - 0 1596763363203 0 connected 10923-16383 2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 172.16.1.51:6379 myself,master - 0 0 1 connected 0-5461 381b54584572e8013becdae2eeaff48bf6eb5450 172.16.1.52:6380 slave acc3a4d0e6e43fc74630c1f0714865fdcbdaf677 0 1596763364211 2 connected
8.故障演示
#停掉一台节点 [root@db03 ~]# reboot #到另一台机器查看集群状态,发现集群是正常的 172.16.1.51:6379> CLUSTER INFO cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:6 cluster_my_epoch:1 cluster_stats_messages_sent:327031 cluster_stats_messages_received:326973 #查看节点信息,副本被提升为主库 172.16.1.51:6379> CLUSTER NODES 5a7f0cf95e1850b5b5ae81d873c4c76fd366d604 172.16.1.51:6380 slave 5eb9e5356534ff4acda736d13f0dc9fc3d40049b 0 1596763771309 5 connected 5eb9e5356534ff4acda736d13f0dc9fc3d40049b 172.16.1.52:6379 master - 0 1596763771310 5 connected 5462-10922 50878ef6a4d8141c8dbca3e2bf7c84ed48a73ee2 172.16.1.53:6380 slave,fail 2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 1596763736458 1596763734245 3 disconnected acc3a4d0e6e43fc74630c1f0714865fdcbdaf677 172.16.1.53:6379 master,fail - 1596763736458 1596763735246 0 disconnected 2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 172.16.1.51:6379 myself,master - 0 0 1 connected 0-5461 381b54584572e8013becdae2eeaff48bf6eb5450 172.16.1.52:6380 master - 0 1596763772319 6 connected 10923-16383
9.节点恢复
#修复机器 [root@db03 ~]# redis-server /service/redis/6379/redis.conf [root@db03 ~]# redis-server /service/redis/6380/redis.conf #再次查看节点信息 172.16.1.51:6379> CLUSTER NODES 5a7f0cf95e1850b5b5ae81d873c4c76fd366d604 172.16.1.51:6380 slave 5eb9e5356534ff4acda736d13f0dc9fc3d40049b 0 1596764061287 5 connected 5eb9e5356534ff4acda736d13f0dc9fc3d40049b 172.16.1.52:6379 master - 0 1596764060781 5 connected 5462-10922 50878ef6a4d8141c8dbca3e2bf7c84ed48a73ee2 172.16.1.53:6380 slave 2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 0 1596764059770 3 connected acc3a4d0e6e43fc74630c1f0714865fdcbdaf677 172.16.1.53:6379 slave 381b54584572e8013becdae2eeaff48bf6eb5450 0 1596764062094 6 connected 2325be6f1f9c1c9f57d5a033fc05e0d798ea823a 172.16.1.51:6379 myself,master - 0 0 1 connected 0-5461 381b54584572e8013becdae2eeaff48bf6eb5450 172.16.1.52:6380 master - 0 1596764061789 6 connected 10923-16383 #原主节点修复后变为从节点
三、使用工具搭建redis集群(方法二)
1.环境准备
节点 | IP | 端口 |
---|---|---|
节点1 | 172.16.1.51 | 6379,6380 |
节点2 | 172.16.1.52 | 6379,6380 |
节点3 | 172.16.1.53 | 6379,6380 |
2.搭建redis
3.启动所有redis
4.安装集群插件
#EPEL源安装ruby支持 [root@db01 ~]# yum install ruby rubygems -y #查看gem源 [root@db01 ~]# gem sources -l *** CURRENT SOURCES *** http://rubygems.org/ #添加阿里云的gem源 [root@db01 ~]# gem sources -a http://mirrors.aliyun.com/rubygems/ http://mirrors.aliyun.com/rubygems/ added to sources #删除国外gem源 [root@db01 ~]# gem sources --remove https://rubygems.org/ http://rubygems.org/ removed from sources #再次查看gem源 [root@db01 ~]# gem sources -l #使用gem安装redis的ruby插件 [root@db01 ~]# gem install redis -v 3.3.3 Successfully installed redis-3.3.3 1 gem installed Installing ri documentation for redis-3.3.3... Installing RDoc documentation for redis-3.3.3...
-
redis-trib.rb命令
[root@db01 ~]# redis-trib.rb create #创建一个集群 check #检查集群 info #集群状态 fix #修复集群 reshard #重新分配槽位 rebalance #平衡槽位数量 add-node #添加节点 del-node #删除节点 set-timeout #设置超时时间 call #向集群所有机器输入命令 import #导入数据 help #帮助
6.关联所有节点
[root@db01 ~]# redis-trib.rb create --replicas 1 172.16.1.51:6379 172.16.1.52:6379 172.16.1.53:6379 172.16.1.52:6380 172.16.1.53:6380 172.16.1.51:6380 # --replicas参数指定集群中每个主节点配备几个从节点,这里设置为1。 >>> Creating cluster >>> Performing hash slots allocation on 6 nodes... Using 3 masters: 172.16.1.51:6379 172.16.1.52:6379 172.16.1.53:6379 Adding replica 172.16.1.52:6380 to 172.16.1.51:6379 Adding replica 172.16.1.51:6380 to 172.16.1.52:6379 Adding replica 172.16.1.53:6380 to 172.16.1.53:6379 M: 5ad7bd957133eac9c3a692b35f8ae72258cf0ece 172.16.1.51:6379 slots:0-5460 (5461 slots) master M: 7c79559b280db9d9c182f3a25c718efe9e934fc7 172.16.1.52:6379 slots:5461-10922 (5462 slots) master M: d27553035a3e91c78d375208c72b756e9b2523d4 172.16.1.53:6379 slots:10923-16383 (5461 slots) master S: fee551a90c8646839f66fa0cd1f6e5859e9dd8e0 172.16.1.52:6380 replicates 5ad7bd957133eac9c3a692b35f8ae72258cf0ece S: e4794215d9d3548e9c514c10626ce618be19ebfb 172.16.1.53:6380 replicates d27553035a3e91c78d375208c72b756e9b2523d4 S: 1d10edbc5ed08f85d2afc21cd338b023b9dd61b4 172.16.1.51:6380 replicates 7c79559b280db9d9c182f3a25c718efe9e934fc7 Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join... >>> Performing Cluster Check (using node 172.16.1.51:6379) M: 5ad7bd957133eac9c3a692b35f8ae72258cf0ece 172.16.1.51:6379 slots:0-5460 (5461 slots) master 1 additional replica(s) S: e4794215d9d3548e9c514c10626ce618be19ebfb 172.16.1.53:6380 slots: (0 slots) slave replicates d27553035a3e91c78d375208c72b756e9b2523d4 M: d27553035a3e91c78d375208c72b756e9b2523d4 172.16.1.53:6379 slots:10923-16383 (5461 slots) master 1 additional replica(s) S: fee551a90c8646839f66fa0cd1f6e5859e9dd8e0 172.16.1.52:6380 slots: (0 slots) slave replicates 5ad7bd957133eac9c3a692b35f8ae72258cf0ece S: 1d10edbc5ed08f85d2afc21cd338b023b9dd61b4 172.16.1.51:6380 slots: (0 slots) slave replicates 7c79559b280db9d9c182f3a25c718efe9e934fc7 M: 7c79559b280db9d9c182f3a25c718efe9e934fc7 172.16.1.52:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s) [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
7.查看集群状态
[root@db01 ~]# redis-cli -h 172.16.1.51 -p 6379 CLUSTER NODES e4794215d9d3548e9c514c10626ce618be19ebfb 172.16.1.53:6380 slave d27553035a3e91c78d375208c72b756e9b2523d4 0 1596767315453 5 connected d27553035a3e91c78d375208c72b756e9b2523d4 172.16.1.53:6379 master - 0 1596767315453 3 connected 10923-16383 5ad7bd957133eac9c3a692b35f8ae72258cf0ece 172.16.1.51:6379 myself,master - 0 0 1 connected 0-5460 fee551a90c8646839f66fa0cd1f6e5859e9dd8e0 172.16.1.52:6380 slave 5ad7bd957133eac9c3a692b35f8ae72258cf0ece 0 1596767313429 4 connected 1d10edbc5ed08f85d2afc21cd338b023b9dd61b4 172.16.1.51:6380 slave 7c79559b280db9d9c182f3a25c718efe9e934fc7 0 1596767313935 6 connected 7c79559b280db9d9c182f3a25c718efe9e934fc7 172.16.1.52:6379 master - 0 1596767314949 2 connected 5461-10922
8.重新做主从
#由于使用工具,始终有一台机器为主库本机的从库,所以要重新分配主从 172.16.1.52:6380> CLUSTER REPLICATE d27553035a3e91c78d375208c72b756e9b2523d4 OK 172.16.1.53:6380> CLUSTER REPLICATE 5ad7bd957133eac9c3a692b35f8ae72258cf0ece OK
9.插入数据测试
[root@db01 ~]# redis-trib.rb info 172.16.1.52:6379 172.16.1.52:6379 (7c79559b...) -> 332 keys | 5462 slots | 1 slaves. 172.16.1.51:6379 (5ad7bd95...) -> 341 keys | 5461 slots | 1 slaves. 172.16.1.53:6379 (d2755303...) -> 327 keys | 5461 slots | 1 slaves. [OK] 1000 keys in 3 masters. 0.06 keys per slot on average. [root@db01 ~]# redis-trib.rb info 172.16.1.52:6379 172.16.1.52:6379 (7c79559b...) -> 661 keys | 5462 slots | 1 slaves. 172.16.1.51:6379 (5ad7bd95...) -> 674 keys | 5461 slots | 1 slaves. 172.16.1.53:6379 (d2755303...) -> 665 keys | 5461 slots | 1 slaves. [OK] 2000 keys in 3 masters. 0.12 keys per slot on average.