一、Redis 集群模式搭建
redis集群部署分为三种:主从、哨兵(Sentinel)、集群(Cluster)。
前几天由于新应用上线,需要部署一套redis集群,现形成文档,将部署过程进行记录。
环境介绍
ip | 软件包 | 系统版本 |
10.1.112.248 | redis-4.0.11.tar.gz | CentOS Linux release 7.6.1810 (Core) |
10.1.112.249 | redis-4.0.11.tar.gz | CentOS Linux release 7.6.1810 (Core) |
1、目录结构
[root@localhost local]# pwd
/usr/local
[root@localhost local]# tree redis-cluster/
redis-cluster/
├── redis-4.0.11-node1
│ ├── data
│ └── logs
├── redis-4.0.11-node2
│ ├── data
│ └── logs
└── redis-4.0.11-node3
├── data
└── logs
9 directories, 0 files
2、下载解压redis-4.0.11
mkdir /usr/local/software
cd /usr/local/software && wget http://download.redis.io/releases/redis-4.0.11.tar.gz
tar xf redis-4.0.11.tar.gz
3、安装(两台服务器均安装)
[root@localhost software]# pwd
/usr/local/software
[root@localhost software]# tar xf redis-4.0.11.tar.gz
[root@localhost software]# cd redis-4.0.1
[root@localhost redis-4.0.11]# make install
出现如下截图即可
4、准备配置文件
[root@localhost redis-4.0.11]# cp src/redis-trib.rb /usr/local/bin/
[root@localhost redis-4.0.11]# cp redis.conf /usr/local/redis-cluster/redis-4.0.11-node1/
[root@localhost redis-4.0.11]# cp redis.conf /usr/local/redis-cluster/redis-4.0.11-node2/
[root@localhost redis-4.0.11]# cp redis.conf /usr/local/redis-cluster/redis-4.0.11-node3/
[root@localhost redis-4.0.11]# cp src/{redis-cli,redis-server} /usr/local/redis-cluster/redis-4.0.11-node1/
[root@localhost redis-4.0.11]# cp src/{redis-cli,redis-server} /usr/local/redis-cluster/redis-4.0.11-node2/
[root@localhost redis-4.0.11]# cp src/{redis-cli,redis-server} /usr/local/redis-cluster/redis-4.0.11-node3/
[root@localhost redis-4.0.11-node1]# grep -Ev "^#|^$" redis.conf
bind 10.1.112.248 #领一台服务器配置为10.1.112.249
protected-mode yes
port 7000 #其他节点分别配置为7001/7002
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize yes
supervised no
pidfile /usr/local/redis-cluster/redis-4.0.11-node1/redis_7000.pid #其他节点配置为redis-4.0.11-node2/redis_7001.pid、redis-4.0.11-node3/redis_7002.pid
loglevel notice
logfile "/usr/local/redis-cluster/redis-4.0.11-node1/logs/redis_7000.log" #同上
databases 16
always-show-logo yes
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir /usr/local/redis-cluster/redis-4.0.11-node1/data #redis-4.0.11-node3/data、redis-4.0.11-node3/data
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
requirepass xxx
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
slave-lazy-flush no
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble no
lua-time-limit 5000
cluster-enabled yes
cluster-config-file nodes-7000.conf
cluster-node-timeout 15000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes
5、分别启动3个节点(另一台服务器同样启动)
cd /usr/local/redis-cluster/redis-4.0.11-node1
./redis.server redis.conf
cd /usr/local/redis-cluster/redis-4.0.11-node2
./redis.server redis.conf
cd /usr/local/redis-cluster/redis-4.0.11-node3
./redis.server redis.conf
查看进程
[root@localhost redis-4.0.11-node1]# ps -ef |grep redis
root 6015 5673 0 17:52 pts/1 00:00:00 grep --color=auto redis
weblogic 28246 1 0 Oct21 ? 00:01:14 ./redis-server 10.1.112.248:7000 [cluster]
weblogic 28284 1 0 Oct21 ? 00:01:14 ./redis-server 10.1.112.248:7001 [cluster]
weblogic 28301 1 0 Oct21 ? 00:02:25 ./redis-server 10.1.112.248:7002 [cluster]
[root@localhost redis-4.0.11-node1]# ps -ef |grep redis
root 6015 5673 0 17:52 pts/1 00:00:00 grep --color=auto redis
weblogic 28246 1 0 Oct21 ? 00:01:14 ./redis-server 10.1.112.249:7000 [cluster]
weblogic 28284 1 0 Oct21 ? 00:01:14 ./redis-server 10.1.112.249:7001 [cluster]
weblogic 28301 1 0 Oct21 ? 00:02:25 ./redis-server 10.1.112.249:7002 [cluster]
6、使用 redis-trib.rb创建集群
[root@localhost redis-4.0.11-node1]# yum -y install ruby ruby-devel rubygems rpm-build
[root@localhost redis-4.0.11-node1]# gem install redis
Fetching: redis-4.1.3.gem (100%)
ERROR: Error installing redis:
redis requires Ruby version >= 2.3.0.
提示ruby版本过低。
升级ruby版本
[root@localhost redis-4.0.11-node1]# curl -L get.rvm.io | bash -s stable
按提示导入秘钥,再次执行"curl -L get.rvm.io | bash -s stable"
[root@localhost redis-4.0.11-node1]# source /usr/local/rvm/scripts/rvm
查看可用的ruby版本并安装
[root@localhost redis-4.0.11-node1]# rvm list known
[root@localhost redis-4.0.11-node1]# rvm install 2.3.0 #这一步用时较长,耐心等待安装完成。
7、安装gem redis接口
rvm use 2.3.0
ruby --version
gem install redis
8、安装rubygems
yum install -y rubygems
至此,Ruby和运行redis-trib.rb需要的环境安装完成了,接下来使用redis-trib.rb创建集群
9、创建集群
redis-trib.rb create --replicas 1 10.1.112.248:7000 10.1.112.248:7001 10.1.112.248:7002 10.1.112.249:7000 10.1.112.249:7001 10.1.112.249:7002
10、查看集群状态
[weblogic@geb-redis1 ~]$ redis-trib.rb check 10.1.112.248:7000
>>> Performing Cluster Check (using node 10.1.112.248:7000)
M: 61d94b047e12679886991c7c6d225b471374a018 10.1.112.248:7000
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: a582b9d7784d2f1b2e6a20d5352a19062dc29d4a 10.1.112.248:7002
slots: (0 slots) slave
replicates 0addc56bce940a659aa2ac4aa6d51677c1f2c523
M: 0addc56bce940a659aa2ac4aa6d51677c1f2c523 10.1.112.249:7000
slots:5461-10922 (5462 slots) master
1 additional replica(s)
S: 087de17dafa03c361d87f7a4c4a5b2d7ce0555fc 10.1.112.249:7001
slots: (0 slots) slave
replicates fd9f572038a7504e088dde150b89beb2bac39e3c
M: fd9f572038a7504e088dde150b89beb2bac39e3c 10.1.112.248:7001
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: 407d6ea2eb3725e142d2e5ae4e1eef4eee1a48ca 10.1.112.249:7002
slots: (0 slots) slave
replicates 61d94b047e12679886991c7c6d225b471374a018
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
11、整体目录结构
[root@geb-redis1 redis-cluster]# tree
.
├── redis-4.0.11-node1
│ ├── data
│ │ ├── appendonly.aof
│ │ └── nodes-7000.conf
│ ├── logs
│ │ └── redis_7000.log
│ ├── redis_7000.pid
│ ├── redis-cli
│ ├── redis.conf
│ └── redis-server
├── redis-4.0.11-node2
│ ├── data
│ │ ├── appendonly.aof
│ │ └── nodes-7001.conf
│ ├── logs
│ │ └── redis_7001.log
│ ├── redis_7001.pid
│ ├── redis-cli
│ ├── redis.conf
│ └── redis-server
└── redis-4.0.11-node3
├── data
│ ├── appendonly.aof
│ └── nodes-7002.conf
├── logs
│ └── redis_7002.log
├── redis_7002.pid
├── redis-cli
├── redis.conf
└── redis-server
9 directories, 21 files
[root@geb-redis2 redis-cluster]# tree
.
├── redis-4.0.11-node1
│ ├── data
│ │ ├── appendonly.aof
│ │ └── nodes-7000.conf
│ ├── logs
│ │ └── redis_7000.log
│ ├── redis_7000.pid
│ ├── redis-cli
│ ├── redis.conf
│ └── redis-server
├── redis-4.0.11-node2
│ ├── data
│ │ ├── appendonly.aof
│ │ └── nodes-7001.conf
│ ├── logs
│ │ └── redis_7001.log
│ ├── redis_7001.pid
│ ├── redis-cli
│ ├── redis.conf
│ └── redis-server
└── redis-4.0.11-node3
├── data
│ ├── appendonly.aof
│ └── nodes-7002.conf
├── logs
│ └── redis_7002.log
├── redis_7002.pid
├── redis-cli
├── redis.conf
└── redis-server
9 directories, 21 files
12、部署过程中需要调整的系统参数
# echo "vm.overcommit_memory = 1" >> /etc/sysctl.conf
# echo "net.core.somaxconn = 2048" >> /etc/sysctl.conf
# cat /etc/rc.local
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi
如果文章对您有帮助,还想了解更过关于k8s相关的实战经验,请微信扫描下方二维码关注“IT运维图谱”公众号或着通过微信搜一搜关注公众号。