redis 集群化 rabbitmq 的基础

最新推荐文章于 2024-10-09 18:09:59 发布

sqhhb12138

最新推荐文章于 2024-10-09 18:09:59 发布

阅读量58

点赞数 3

文章标签： redis rabbitmq

本文链接：https://blog.csdn.net/sqhhb12138/article/details/134298782

版权

redis的集群化

redis的集群化配置过程中以本地使用的3台虚拟机为例子

3台虚拟机均配置好最基础的redis数据库 redis的版本最好在5.0以上 redis在3.0时开始支持集群化的配置而在5.0时集成到安装包中

在安装好redis的机器中创建出6个用来配置集群的redis的实例在相应的实例中写入相应的配置文件

port 7001 #实例要开放的端口
cluster-enabled yes   #开启集群化
cluster-config-file nodes.conf #集群的配置文件
cluster-node-timeout 5000    #集群化的超时时间
appendonly yes                     #AOF持久化
daemonize yes                      #后台运行
bind 0.0.0.0                            #监听的端口
pidfile  /data/application/7001/redis_7001.pid         #实例开启时需要的pid号存放的文件
dir  /data/application/7001/data                                #实例的数据存放目录

在相应的文件要有相应的目录存在 dir的目录 /data/application/7001/data 目录

接下来启动实例

[root@localhost application]# ./redis-5.0.0/src/redis-server 7001/redis.conf

使用上述命令依次启动实例

启动完成后

在任意一个虚拟机中使用

[root@localhost application]# ./redis-5.0.0/src/redis-cli --cluster create --cluster-replicas 1 192.168.75.145:7001 192.168.75.145:7002 192.168.75.142:7003 192.168.75.142:7004 192.168.75.143:7005 192.168.75.143:7006

其中create为创建一个集群 --cluster-replicas 后面跟着的便是这些要加入集群的实例

依次往下即可根据提示自动分配实例的集群位置

其中任意一台设置了密码那其他的实例最好也设置成为相同的密码

在登陆时使用

[root@localhost application]# ./redis-5.0.0/src/redis-cli -c -p 7001

如果使用了密码则在登陆时使用 “-a password ”“ password为设置的密码进行登录

此时登录任意一台集群里的实例都可以看到整个集群里的情况

[root@localhost src]# ./redis-cli -p 7003
127.0.0.1:7003> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:5
cluster_size:3
cluster_current_epoch:7
cluster_my_epoch:7
cluster_stats_messages_ping_sent:732
cluster_stats_messages_pong_sent:773
cluster_stats_messages_meet_sent:2
cluster_stats_messages_fail_sent:8
cluster_stats_messages_sent:1515
cluster_stats_messages_ping_received:770
cluster_stats_messages_pong_received:723
cluster_stats_messages_meet_received:3
cluster_stats_messages_fail_received:3
cluster_stats_messages_auth-req_received:1
cluster_stats_messages_received:1500
127.0.0.1:7003> cluster nodes
8b98f03c2855334b733d1928e42e1bd12ab7d6bd 192.168.75.145:7001@17001 master - 0 1699329747121 1 connected 0-5460
a1236b2fdb59e96a1d102a16244f036db300ac7d 192.168.75.145:7002@17002 slave a79d6ff27f2b2f7f647da300ff56ac90408ddceb 0 1699329748095 5 connected
a79d6ff27f2b2f7f647da300ff56ac90408ddceb 192.168.75.143:7005@17005 master - 0 1699329748000 5 connected 10923-16383
2b307aad3e3e974f94704225059d6e49452c879c 192.168.75.143:7006@17006 master - 0 1699329748198 7 connected 5461-10922
8a272a9eab4cdbde8d297bdfb0744bc46ee64c05 192.168.75.142:7003@17003 myself,slave 2b307aad3e3e974f94704225059d6e49452c879c 0 1699329747000 3 connected

其中使用 cluster info 可以看到集群的基本io

使用 cluster nodes 可以看到整个集群的配置情况前面的为各个实力的标识码后面带着 0-5460 的槽点的为主机整个集群总共平分 16383 个槽点

可以根据实际情况进行增加或者删除

添加实例与释放实例

首先进行添加实例

启动实例之后进行

[root@localhost src]# ./redis-cli --cluster add-node 192.168.75.142:7004 192.168.75.142:7003

其中前面的部分为执行的指令部分后面跟着的第一个参数为添加的实例后面跟着的是集群内的实例

>>> Adding node 192.168.75.142:7004 to cluster 192.168.75.142:7003
>>> Performing Cluster Check (using node 192.168.75.142:7003)
S: 8a272a9eab4cdbde8d297bdfb0744bc46ee64c05 192.168.75.142:7003
   slots: (0 slots) slave
   replicates 2b307aad3e3e974f94704225059d6e49452c879c
M: 8b98f03c2855334b733d1928e42e1bd12ab7d6bd 192.168.75.145:7001
   slots:[0-5460] (5461 slots) master
S: a1236b2fdb59e96a1d102a16244f036db300ac7d 192.168.75.145:7002
   slots: (0 slots) slave
   replicates a79d6ff27f2b2f7f647da300ff56ac90408ddceb
M: a79d6ff27f2b2f7f647da300ff56ac90408ddceb 192.168.75.143:7005
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
M: 2b307aad3e3e974f94704225059d6e49452c879c 192.168.75.143:7006
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.75.142:7004 to make it join the cluster.
[OK] New node added correctly.

显示如上信息时显示添加成功

通过新添加的实例登入到redis中查看集群的状态

[root@localhost src]# ./redis-cli -p 7004
127.0.0.1:7004>cluster nodes
8b98f03c2855334b733d1928e42e1bd12ab7d6bd 192.168.75.145:7001@17001 master - 0 1699329990011 1 connected 0-5460
a1236b2fdb59e96a1d102a16244f036db300ac7d 192.168.75.145:7002@17002 slave a79d6ff27f2b2f7f647da300ff56ac90408ddceb 0 1699329991125 5 connected
a79d6ff27f2b2f7f647da300ff56ac90408ddceb 192.168.75.143:7005@17005 master - 0 1699329990575 5 connected 10923-16383
2b307aad3e3e974f94704225059d6e49452c879c 192.168.75.143:7006@17006 master - 0 1699329990575 7 connected 5461-10922
d1ff6d7177478f94dfdda0bd8aaebf956c37b659 192.168.75.142:7004@17004 master - 0 1699329988860 0 connected
8a272a9eab4cdbde8d297bdfb0744bc46ee64c05 192.168.75.142:7003@17003 myself,slave 2b307aad3e3e974f94704225059d6e49452c879c 0 1699329986000 3 connected

可以看出新添加的 ”d1ff6d7177478f94dfdda0bd8aaebf956c37b659 192.168.75.142:7004@17004 master - 016993299888600 connected

“实例并没有分配到槽点并且是一台主机

接着执行如下命令将先添加的实例添加为单个带有槽点的主机的从机

127.0.0.1:7004> cluster replicate 8b98f03c2855334b733d1928e42e1bd12ab7d6bd
OK

重新进行查看

127.0.0.1:7004> cluster nodes
a1236b2fdb59e96a1d102a16244f036db300ac7d 192.168.75.145:7002@17002 slave a79d6ff27f2b2f7f647da300ff56ac90408ddceb 0 1699330241650 5 connected
2b307aad3e3e974f94704225059d6e49452c879c 192.168.75.143:7006@17006 master - 0 1699330240439 7 connected 5461-10922
a79d6ff27f2b2f7f647da300ff56ac90408ddceb 192.168.75.143:7005@17005 master - 0 1699330241000 5 connected 10923-16383
d1ff6d7177478f94dfdda0bd8aaebf956c37b659 192.168.75.142:7004@17004 myself,slave 8b98f03c2855334b733d1928e42e1bd12ab7d6bd 0 1699330237000 0 connected
8b98f03c2855334b733d1928e42e1bd12ab7d6bd 192.168.75.145:7001@17001 master - 0 1699330242085 1 connected 0-5460
8a272a9eab4cdbde8d297bdfb0744bc46ee64c05 192.168.75.142:7003@17003 slave 2b307aad3e3e974f94704225059d6e49452c879c 0 1699330241530 7 connected

可以看出配置成功

删除一个集群节点

与添加相对执行命令时 add-node 更改为 del-node 即可

[root@localhost src]# ./redis-cli --cluster del-node 192.168.75.142:7004 d1ff6d7177478f94dfdda0bd8aaebf956c37b659
>>> Removing node d1ff6d7177478f94dfdda0bd8aaebf956c37b659 from cluster 192.168.75.142:7004
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

登录到集群中可以看到

[root@localhost src]# ./redis-cli -p 7003
127.0.0.1:7003> cluster nodes
8b98f03c2855334b733d1928e42e1bd12ab7d6bd 192.168.75.145:7001@17001 master - 0 1699352268930 1 connected 0-5460
a1236b2fdb59e96a1d102a16244f036db300ac7d 192.168.75.145:7002@17002 slave a79d6ff27f2b2f7f647da300ff56ac90408ddceb 0 1699352267823 5 connected
a79d6ff27f2b2f7f647da300ff56ac90408ddceb 192.168.75.143:7005@17005 master - 0 1699352268149 5 connected 10923-16383
2b307aad3e3e974f94704225059d6e49452c879c 192.168.75.143:7006@17006 master - 0 1699352269254 7 connected 5461-10922
8a272a9eab4cdbde8d297bdfb0744bc46ee64c05 192.168.75.142:7003@17003 myself,slave 2b307aad3e3e974f94704225059d6e49452c879c 0 1699352167000 3 connected

其中已经没有删除的那个节点的信息

而要删除主节点则需要先将该主节点的槽点释放到其他的节点中

[root@localhost src]# ./redis-cli --cluster reshard 192.168.75.145:7001
>>> Performing Cluster Check (using node 192.168.75.145:7001)
M: 8b98f03c2855334b733d1928e42e1bd12ab7d6bd 192.168.75.145:7001
   slots:[2729-5460] (2732 slots) master
M: a79d6ff27f2b2f7f647da300ff56ac90408ddceb 192.168.75.143:7005
   slots:[0-2728],[5461-8191],[10923-16383] (10921 slots) master
   1 additional replica(s)
M: 2b307aad3e3e974f94704225059d6e49452c879c 192.168.75.143:7006
   slots:[8192-10922] (2731 slots) master
   1 additional replica(s)
S: a1236b2fdb59e96a1d102a16244f036db300ac7d 192.168.75.145:7002
   slots: (0 slots) slave
   replicates a79d6ff27f2b2f7f647da300ff56ac90408ddceb
S: 8a272a9eab4cdbde8d297bdfb0744bc46ee64c05 192.168.75.142:7003
   slots: (0 slots) slave
   replicates 2b307aad3e3e974f94704225059d6e49452c879c
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 600
#此为输入需要移动的槽点的数量
What is the receiving node ID? 2b307aad3e3e974f94704225059d6e49452c879c
#此为接受的实例的id （集群中分配的）
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1: 8b98f03c2855334b733d1928e42e1bd12ab7d6bd
#此为释放槽点的也就是需要删除的主节点的id
Source node #2: done

此后再次进行查看

[root@localhost src]# ./redis-cli -p 7003
127.0.0.1:7003> cluster nodes
8b98f03c2855334b733d1928e42e1bd12ab7d6bd 192.168.75.145:7001@17001 master - 0 1699353077768 1 connected
a1236b2fdb59e96a1d102a16244f036db300ac7d 192.168.75.145:7002@17002 slave a79d6ff27f2b2f7f647da300ff56ac90408ddceb 0 1699353076356 8 connected
a79d6ff27f2b2f7f647da300ff56ac90408ddceb 192.168.75.143:7005@17005 master - 0 1699353077445 8 connected 0-2728 5461-8191 10923-16383
2b307aad3e3e974f94704225059d6e49452c879c 192.168.75.143:7006@17006 master - 0 1699353076356 9 connected 2729-5460 8192-10922
8a272a9eab4cdbde8d297bdfb0744bc46ee64c05 192.168.75.142:7003@17003 myself,slave 2b307aad3e3e974f94704225059d6e49452c879c 0 1699352167000 3 connected

可以看到要删除的主节点的槽点为空了就可以进行节点的删除了

[root@localhost src]# ./redis-cli --cluster del-node 192.168.75.145:7001 8b98f03c2855334b733d1928e42e1bd12ab7d6bd
>>> Removing node 8b98f03c2855334b733d1928e42e1bd12ab7d6bd from cluster 192.168.75.145:7001
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

rabbitmq基础

rabbitmq是一款消息中间件旨在减少数据库写的压力通过消息队列的方式将一部分已经决定要写数据库的消息放在队列中等待数据库进行写入操作；其中并不会影响到客户的使用

首先下载所需要的依赖

[root@localhost ~]# yum install -y epel-release gcc-c++ unixODBC unixODBC-devel openssl-devel ncurses-devel

[root@localhost ~]# curl -s https://packagecloud.io/install/repositories/rabbitmq/erlang/script.rpm.sh | sudo bash

下载erlang的环境

[root@localhost ~]# rpm -ivh erlang-21.3.8.21-1.el7.x86_64.rpm 


[root@localhost ~]# erl     
Erlang/OTP 21 [erts-10.3.5.16] [source] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:1] [hipe]

Eshell V10.3.5.16  (abort with ^G)
1> 
BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded
       (v)ersion (k)ill (D)b-tables (d)istribution

可以通过查看erl 命令查看是否安装成功

接下直接安装rabbitmq

[root@localhost ~]# rpm -ivh rabbitmq-server-3.7.10-1.el7.noarch.rpm 
警告：rabbitmq-server-3.7.10-1.el7.noarch.rpm: 头V4 RSA/SHA1 Signature, 密钥 ID 6026dfca: NOKEY
错误：依赖检测失败：
        socat 被 rabbitmq-server-3.7.10-1.el7.noarch 需要
#下载所需的依赖socat后再次进行安装操作      
[root@localhost ~]# yum -y install socat

[root@localhost ~]# systemctl daemon-reload   #环境变量重载
[root@localhost ~]# systemctl start rabbitmq-server #启动
[root@localhost ~]# /sbin/service rabbitmq-server status  #查看状态
[root@localhost rabbitmq]# rabbitmq-plugins enable rabbitmq_management  #打开网页状态查看

[root@localhost ~]# rabbitmqctl add_user rabbit 123456 #添加用户以及密码
[root@localhost ~]# rabbitmqctl set_user_tags rabbit administrator #设置用户为管理员

此处设置权限时注意'.*'之间需要有空格 三个'.*'分别代表了conf权限，read权限与write权限 例如：当没有给
newrain设置这三个权限前是没有权限查询队列，在ui界面也看不见
[root@rabbitmq-1 ~]# rabbitmqctl set_permissions -p "/" newrain ".*" ".*" ".*"
[root@localhost ~]# rabbitmqctl list_users   #列出用户
[root@localhost rabbitmq]# cp /usr/share/doc/rabbitmq-server-3.7.10/rabbitmq.config.example  .

[root@localhost rabbitmq]# vim rabbitmq.config  修改其中的/loop使其能够远程登陆

测试

使用第三方工具

[root@localhost rabbitmq-test]# cat README.md

# 该项目是rabbitmq的测试项目

> 开发者: newrain

```shell

使用方法：

git clone https://gitea.beyourself.org.cn/newrain001/rabbitmq-test.git && \

cd rabbitmq-test && yum install -y python3 python3-devel && \

pip3 install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple && \

export FLASK_ENV=development ; flask run --reload -p 80

按照命令执行最后执行

[root@localhost rabbitmq-test]# flask run --reload -p 80 -h 0.0.0.0
 * Environment: development
 * Debug mode: on
 * Running on all addresses.
   WARNING: This is a development server. Do not use it in a production deployment.
 * Running on http://192.168.75.145:80/ (Press CTRL+C to quit)
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 587-179-852
192.168.75.1 - - [02/Nov/2023 15:30:07] "GET / HTTP/1.1" 200 -
192.168.75.1 - - [02/Nov/2023 15:30:07] "GET /favicon.ico HTTP/1.1" 404 -
192.168.75.1 - - [02/Nov/2023 15:30:46] "POST / HTTP/1.1" 200 -

在网页上打开指定的ip 输入测试需要的消息数量即可对rabbitmq进行测试