CentOS 7.4下Redis及集群的安装及配置

最新推荐文章于 2024-11-13 22:17:16 发布

weixin_33805992

最新推荐文章于 2024-11-13 22:17:16 发布

阅读量136

点赞数

文章标签：数据库 ruby 网络

原文链接：http://blog.51cto.com/liqingbiao/2177019

版权

（一）Redis概述
Redis 是一个开源、支持网络、基于内存、键值对的 Key-Value 数据库，使用 ANSI C 编写，并提供多种语言的 API ，它几乎没有上手难度，只需要几分钟我们就能完成安装工作，并让它开始与应用程序顺畅协作。换句话来说，只需投入一小部分时间与精力，大家就能获得立竿见影且效果极佳的性能表现提升，就是说它是一个非常简单缓存解决方案。它支持存储的 Value 类型不仅限于字符串，支持主从同步，数据持久化等等.
Redis 是一个开源的 key-value 存储系统，由于出众的性能，大部分互联网企业都用来做服务器端缓存。Redis 在3.0版本前只支持单实例模式，虽然支持主从模式、哨兵模式部署来解决单点故障，但是现在互联网企业动辄大几百G的数据，可完全是没法满足业务的需求，所以，Redis 在 3.0 版本以后就推出了集群模式。
Redis 3.0集群采用了P2P的模式，完全去中心化。Redis把所有的Key分成了16384个slot，每个Redis实例负责其中一部分slot。集群中的所有信息（节点、端口、slot等），都通过节点之间定期的数据交换而更新。Redis客户端在任意一个Redis实例发出请求，如果所需数据不在该实例中，通过重定向命令引导客户端访问所需的实例。

总的来说就是多个实例组成了集群，数据分散在不同的实例上，然后你可以随便去哪个实例都能读到数据。另外为了保证高可用，还要配置从节点。

（二）安装及配置

[root@ELK-Redis2 ~]#yum install -y gcc c++ jemalloc
[root@ELK-Redis2 ~]#wget http://download.redis.io/releases/redis-4.0.10.tar.gz
[root@ELK-Redis2 ~]#tar xf redis-4.0.10.tar.gz
[root@ELK-Redis2 ~]#cd redis-4.0.10
[root@ELK-Redis2 ~]#make && make install
[root@ELK-Redis2 ~]#cp utils/redis_init_script /etc/init.d/redis
[root@ELK-Redis2 ~]#chmod +x /etc/init.d/redis
[root@ELK-Redis2 ~]#mkdir -p /etc/redis/ && mkdir -p /var/log/redis/ && mkdir -p /data/redis_db
[root@ELK-Redis2 ~]#cp redis.conf /etc/redis/6379.conf
[root@ELK-Redis2 ~]#sed -i '/daemonize no/s/no/yes/' /etc/redis/6379.conf
[root@ELK-Redis2 ~]#sed -i '/^pidfile/s/redis.pid/redis_6379.pid/' /etc/redis/6379.conf
[root@ELK-Redis2 ~]#sed -i '/^logfile/s/""/"\/var\/log\/redis\/redis.log"/' /etc/redis/6379.conf
[root@ELK-Redis2 ~]#sed -i '/^dir/s/.\//\/data\/redis_db/' /etc/redis/6379.conf
[root@ELK-Redis2 ~]#sed -i '/^slave-read-only/s/yes/no/' /etc/redis/6379.conf
[root@ELK-Redis2 ~]#sed -i '/^# maxclients/s/# //;/^maxclients/s/10000/100000/' /etc/redis/6379.conf

备注如果连接出错的话“
18180:M 20 Sep 18:10:35.061 * 10 changes in 300 seconds. Saving...
18180:M 20 Sep 18:10:35.061 # Can't save in background: fork: Cannot al”
彻底解决的办法是：
1、vim /etc/sysctl.conf
2、添加 vm.overcommit_memory=1
3、sysctl -p

vm.overcommit_memory 这个参数又是干什么的呢？
Linux对大部分申请内存的请求都回复"yes"，以便能跑更多更大的程序。因为申请内存后，并不会马上使用内存，将这些不会使用的空闲内存分配给其它程序使用，以提高内存利用率，这种技术叫做Overcommit。一般情况下，当所有程序都不会用到自己申请的所有内存时，系统不会出问题，但是如果程序随着运行，需要的内存越来越大，在自己申请的大小范围内，不断占用更多内存，直到超出物理内存，当linux发现内存不足时，会发生OOM killer(OOM=out-of-memory)。它会选择杀死一些进程(用户态进程，不是内核线程，哪些占用内存越多，运行时间越短的进程越有可能被杀掉)，以便释放内存。

（三）redis集群的搭建
Redis支持集群最小的单位为6个实例，3个主节点，3个从节点
集群搭建：至少要三个master
两个节点：
master:172.20.144.105 （172.20.144.105:6377 ，172.20.144.105:6378 ，172.20.144.105:6379）
slave: 172.20.144.106 （172.20.144.106:6377 ,172.20.144.106:6378 , 172.20.144.106:6379）

（1）创建相关redis的实例的相关配置
1.1、redis安装及相关配置

[root@ELK-Redis1 ~]#tar xf redis-4.0.10.tar.gz
[root@ELK-Redis1 ~]#cd redis-4.0.10
[root@ELK-Redis1 ~]#make && make install
[root@ELK-Redis1 ~]#cp utils/redis_init_script /etc/init.d/redis
[root@ELK-Redis1 ~]#chmod +x /etc/init.d/redis
[root@ELK-Redis1 ~]#mkdir -p /etc/redis/ && mkdir -p /var/log/redis/ && mkdir -p /data/redis_db
[root@ELK-Redis1 redis-4.0.10]#mkdir /etc/redis/{6377,6378,6379}
[root@ELK-Redis1 redis-4.0.10]# cp redis.conf /etc/redis/6377/redis.conf
[root@ELK-Redis1 redis-4.0.10]# cp redis.conf /etc/redis/6378/redis.conf
[root@ELK-Redis1 redis-4.0.10]# cp redis.conf /etc/redis/6379/redis.conf
[root@ELK-Redis1 redis-4.0.10]# sed -i '/^dir/s/.\//\/data\/redis_db/' /etc/redis/6377/redis.conf 
[root@ELK-Redis1 redis-4.0.10]# sed -i '/^dir/s/.\//\/data\/redis_db/' /etc/redis/6378/redis.conf 
[root@ELK-Redis1 redis-4.0.10]# sed -i '/^dir/s/.\//\/data\/redis_db/' /etc/redis/6379/redis.conf

1.2、依次修改：172.20.144.105:6377、172.20.144.105:6378、172.20.144.105:6379,172.20.144.106:6377、172.20.144.106:6378、172.20.144.106:6379的相关的配置文件

[root@ELK-Redis1 redis-4.0.10]# grep '^[a-Z]' /etc/redis/6377/redis.conf 
bind 172.20.144.105
protected-mode yes
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize yes
supervised no
pidfile /var/run/redis_6379.pid
loglevel notice
logfile ""
databases 16
always-show-logo yes
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir /data/redis_db
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
slave-lazy-flush no
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble no
lua-time-limit 5000
cluster-enabled yes
cluster-config-file nodes-6379.conf
cluster-node-timeout 15000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes

修改相关文件的配置
port  7000                                        //端口7000,7001,7002        
bind 本机ip                                       //默认ip为127.0.0.1 需要改为其他节点机器可访问的ip 否则创建集群时无法访问对应的端口，无法创建集群
daemonize    yes                               //redis后台运行
pidfile  /var/run/redis_7000.pid          //pidfile文件对应7000,7001,7002
cluster-enabled  yes                           //开启集群  把注释#去掉
cluster-config-file  nodes_7000.conf   //集群的配置  配置文件首次启动自动生成 7000,7001,7002
cluster-node-timeout  15000                //请求超时  默认15秒，可自行设置
appendonly  yes                           //aof日志开启  有需要就开启，它会每次写操作都记录一条日志

1.3、依次启动主配置文件

[root@ELK-Redis1 redis-4.0.10]# redis-server /etc/redis/6377/redis.conf 
1329:C 18 Sep 20:33:59.546 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1329:C 18 Sep 20:33:59.546 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=1329, just started
1329:C 18 Sep 20:33:59.546 # Configuration loaded
[root@ELK-Redis1 redis-4.0.10]# redis-server /etc/redis/6378/redis.conf 
1340:C 18 Sep 20:34:06.981 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1340:C 18 Sep 20:34:06.981 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=1340, just started
1340:C 18 Sep 20:34:06.981 # Configuration loaded
[root@ELK-Redis1 redis-4.0.10]# redis-server /etc/redis/6379/redis.conf 
1348:C 18 Sep 20:34:10.703 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1348:C 18 Sep 20:34:10.703 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=1348, just started
1348:C 18 Sep 20:34:10.703 # Configuration loaded
[root@ELK-Redis1 redis-4.0.10]# netstat -lntp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 172.20.144.105:6379     0.0.0.0:*               LISTEN      1349/redis-server 1 
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1189/sshd           
tcp        0      0 127.0.0.1:16377         0.0.0.0:*               LISTEN      1330/redis-server 1 
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1455/master         
tcp        0      0 172.20.144.105:16378    0.0.0.0:*               LISTEN      1341/redis-server 1 
tcp        0      0 172.20.144.105:16379    0.0.0.0:*               LISTEN      1349/redis-server 1 
tcp        0      0 0.0.0.0:10050           0.0.0.0:*               LISTEN      1249/zabbix_agentd  
tcp        0      0 127.0.0.1:6377          0.0.0.0:*               LISTEN      1330/redis-server 1 
tcp        0      0 172.20.144.105:6378     0.0.0.0:*               LISTEN      1341/redis-server 1 
tcp6       0      0 :::22                   :::*                    LISTEN      1189/sshd           
tcp6       0      0 ::1:25                  :::*                    LISTEN      1455/master

[root@ELK-Redis2 redis-4.0.10]# redis-server /etc/redis/6377/redis.conf 
6675:C 18 Sep 21:03:59.530 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
6675:C 18 Sep 21:03:59.530 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=6675, just started
6675:C 18 Sep 21:03:59.530 # Configuration loaded
[root@ELK-Redis2 redis-4.0.10]# redis-server /etc/redis/6378/redis.conf 
6685:C 18 Sep 21:04:05.334 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
6685:C 18 Sep 21:04:05.334 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=6685, just started
6685:C 18 Sep 21:04:05.334 # Configuration loaded
[root@ELK-Redis2 redis-4.0.10]# redis-server /etc/redis/6379/redis.conf 
6722:C 18 Sep 21:04:39.916 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
6722:C 18 Sep 21:04:39.916 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=6722, just started
6722:C 18 Sep 21:04:39.916 # Configuration loaded
[root@ELK-Redis2 redis-4.0.10]# netstat -lntp|grep 637*
tcp        0      0 172.20.144.106:6379     0.0.0.0:*               LISTEN      6723/redis-server 1 
tcp        0      0 172.20.144.106:16377    0.0.0.0:*               LISTEN      6676/redis-server 1 
tcp        0      0 172.20.144.106:16378    0.0.0.0:*               LISTEN      6686/redis-server 1 
tcp        0      0 172.20.144.106:16379    0.0.0.0:*               LISTEN      6723/redis-server 1 
tcp        0      0 172.20.144.106:6377     0.0.0.0:*               LISTEN      6676/redis-server 1 
tcp        0      0 172.20.144.106:6378     0.0.0.0:*               LISTEN      6686/redis-server 1 
[root@ELK-Redis2 redis-4.0.10]# ps -ef|grep redis
root      6676     1  0 21:03 ?        00:00:00 redis-server 172.20.144.106:6377 [cluster]
root      6686     1  0 21:04 ?        00:00:00 redis-server 172.20.144.106:6378 [cluster]
root      6723     1  0 21:04 ?        00:00:00 redis-server 172.20.144.106:6379 [cluster]
root      6791 24114  0 21:05 pts/0    00:00:00 grep --color=auto redis

至此，其实还不是集群，只是6个redis 实例，下一步我们来创建集群

（2）创建集群
2.1、首先安装ruby

[root@ELK-Redis2 redis-4.0.10]# yum -y install ruby ruby-devel rubygems rpm-build
[root@ELK-Redis2 redis-4.0.10]# gem install redis
ERROR:  Could not find a valid gem 'redis' (>= 0), here is why:
          Unable to download data from https://rubygems.org/ - no such name (https://rubygems.org/latest_specs.4.8.gz)
由于没有连接外网，所以可以先下载下来然后在安装，相关链接（https://rubygems.org/downloads/redis-3.3.5.gem）
其他版本的文件：https://rubygems.org/gems/redis/versions

[root@ELK-Redis2 opt]# gem install -l /opt/redis-3.3.5.gem 
Successfully installed redis-3.3.5
Parsing documentation for redis-3.3.5
Installing ri documentation for redis-3.3.5
1 gem installed

2.2、创建集群。redis-trib.rb是redis官方推出的管理redis集群的工具，集成在redis的源码src目录下，是基于redis提供的集群命令封装成简单、便捷、实用的操作工具。redis-trib.rb是redis作者用ruby完成的

[root@ELK-Redis2 opt]# cp /opt/redis-4.0.10/src/redis-trib.rb /usr/local/bin/     
[root@ELK-Redis1 src]# redis-trib.rb create  --replicas  1  172.20.144.105:6377 172.20.144.105:6378 172.20.144.105:6379 172.20.144.106:6377 172.20.144.106:6378 172.20.144.106:6379
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
172.20.144.105:6377
172.20.144.106:6377
172.20.144.105:6378
Adding replica 172.20.144.106:6379 to 172.20.144.105:6377
Adding replica 172.20.144.105:6379 to 172.20.144.106:6377
Adding replica 172.20.144.106:6378 to 172.20.144.105:6378
M: 0905078cce6cf980b9362e7d9b6cb7e6ad1d874c 172.20.144.105:6377
   slots:0-5460 (5461 slots) master
M: 31ef8f27b2fb9fdebbbd562c2c4e589c42f83066 172.20.144.105:6378
   slots:10923-16383 (5461 slots) master
S: dc1b2174b49363d8cbd7276bb25bbcd663b358c7 172.20.144.105:6379
   replicates cb652e954181aeecb27bb2b1a6f06e8c31d49d4a
M: cb652e954181aeecb27bb2b1a6f06e8c31d49d4a 172.20.144.106:6377
   slots:5461-10922 (5462 slots) master
S: cb79c3c8520cfb6dbbd685e2289673d9425bd942 172.20.144.106:6378
   replicates 31ef8f27b2fb9fdebbbd562c2c4e589c42f83066
S: a86950a77fd9fb387c246938c6c25d553cce4bcb 172.20.144.106:6379
   replicates 0905078cce6cf980b9362e7d9b6cb7e6ad1d874c
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join......
>>> Performing Cluster Check (using node 172.20.144.105:6377)
M: 0905078cce6cf980b9362e7d9b6cb7e6ad1d874c 172.20.144.105:6377
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: cb79c3c8520cfb6dbbd685e2289673d9425bd942 172.20.144.106:6378
   slots: (0 slots) slave
   replicates 31ef8f27b2fb9fdebbbd562c2c4e589c42f83066
S: a86950a77fd9fb387c246938c6c25d553cce4bcb 172.20.144.106:6379
   slots: (0 slots) slave
   replicates 0905078cce6cf980b9362e7d9b6cb7e6ad1d874c
M: 31ef8f27b2fb9fdebbbd562c2c4e589c42f83066 172.20.144.105:6378
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: dc1b2174b49363d8cbd7276bb25bbcd663b358c7 172.20.144.105:6379
   slots: (0 slots) slave
   replicates cb652e954181aeecb27bb2b1a6f06e8c31d49d4a
M: cb652e954181aeecb27bb2b1a6f06e8c31d49d4a 172.20.144.106:6377
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

备注：单解释一下这个命令：调用 ruby 命令来进行创建集群，--replicas 1 表示主从复制比例为 1:1，即一个主节点对应一个从节点；然后，默认给我们分配好了每个主节点和对应从节点服务，以及 solt 的大小，因为在 Redis 集群中有且仅有 16383 个 solt ，默认情况会给我们平均分配，当然你可以指定，后续的增减节点也可以重新分配。

2.3、验证集群节点

[root@ELK-Redis1 src]# redis-trib.rb check 172.20.144.106:6379
>>> Performing Cluster Check (using node 172.20.144.106:6379)
S: a86950a77fd9fb387c246938c6c25d553cce4bcb 172.20.144.106:6379
   slots: (0 slots) slave
   replicates 0905078cce6cf980b9362e7d9b6cb7e6ad1d874c
M: 0905078cce6cf980b9362e7d9b6cb7e6ad1d874c 172.20.144.105:6377
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: 31ef8f27b2fb9fdebbbd562c2c4e589c42f83066 172.20.144.105:6378
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
M: cb652e954181aeecb27bb2b1a6f06e8c31d49d4a 172.20.144.106:6377
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: cb79c3c8520cfb6dbbd685e2289673d9425bd942 172.20.144.106:6378
   slots: (0 slots) slave
   replicates 31ef8f27b2fb9fdebbbd562c2c4e589c42f83066
S: dc1b2174b49363d8cbd7276bb25bbcd663b358c7 172.20.144.105:6379
   slots: (0 slots) slave
   replicates cb652e954181aeecb27bb2b1a6f06e8c31d49d4a
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

2.4、redis-trib.rb 相关功能，前面在创建redis集群的时候已经用过了，这是redis官方给出的一个
集群管理工具。它具有以下功能

1create：创建集群
2check：检查集群
3info：查看集群信息
4fix：修复集群
5reshard：在线迁移slot
6rebalance：平衡集群节点slot数量
7add-node：将新节点加入集群
8del-node：从集群中删除节点
9set-timeout：设置集群节点间心跳连接的超时时间
10call：在集群全部节点上执行命令
11import：将外部redis数据导入集群

转载于:https://blog.51cto.com/liqingbiao/2177019