要让 Redis3.0 集群正常工作至少需要 3 个 Master 节点,要想实现高可用,每个 Master 节点要配备
至少 1 个 Slave 节点。根据以上特点和要求,进行如下的集群实施规划:
使用 2台服务器(物理机或虚拟机)部署 3 个 Master + 3 个 Slave;如果你服务器多的话可以 选择6台,每个redisserver 安装在不同服务器下。
主机名 IP 服务器端口 集群端口 主/从
redis1 192.168.1.102 7111 17111 Master
redis2 192.168.1.102 7112 17112 Master
redis3 192.168.1.102 7113 17113 Master
redis4 192.168.1.105 7114 17114 Master
redis5 192.168.1.105 7115 17115 Master
redis6 192.168.1.105 7116 17116 Master
按规划:防火墙中打开相应的端口 或者 直接关掉防火墙。
192.168.1.102
-A INPUT -m state --state NEW -m tcp -p tcp --dport 7111 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 17111 -j ACCEPT
192.168.1.102
-A INPUT -m state --state NEW -m tcp -p tcp --dport 7112 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 17112 -j ACCEPT
192.168.1.102
-A INPUT -m state --state NEW -m tcp -p tcp --dport 7113 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 17113 -j ACCEPT
192.168.1.105
-A INPUT -m state --state NEW -m tcp -p tcp --dport 7114 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 17114 -j ACCEPT
192.168.1.105
-A INPUT -m state --state NEW -m tcp -p tcp --dport 7115 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 17115 -j ACCEPT
192.168.1.105
-A INPUT -m state --state NEW -m tcp -p tcp --dport 7116 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 17116 -j ACCEPT
安装目录,分别安装在 192.168.1.102服务器目录 /usr/local/redis301 /usr/local/redis302 /usr/local/redis303 和服务器192.168.1.105目录 /usr/local/redis304 /usr/local/redis305 /usr/local/redis306
用户:root
1).编译和安装所需的包:
yum 安装 gcc 和 tcl 包:
# yum install gcc tcl
2).下载(或上传)Redis3 最新稳定版(当前最新版 redis-3.0.2.tar.gz)
# cd /usr/local/
# wget http://download.redis.io/releases/redis-3.0.2.tar.gz
3). 分别在每个服务器创建安装目录:
服务器:192.168.1.102下
# mkdir /usr/local/redis301
# mkdir /usr/local/redis302
# mkdir /usr/local/redis303
服务器:192.168.1.105下
# mkdir /usr/local/redis304
# mkdir /usr/local/redis305
# mkdir /usr/local/redis306
4).解压:
首先是 192.168.1.102 服务器
进入redis安装包下载的所在目录,这里我下载到了 /usr/local 目录下:
#cd /usr/local
# tar -zxvf redis-3.0.2.tar.gz
会产生解压缩文件:
redis-3.0.2
5). 安装(使用 PREFIX 指定安装目录):
首先进入解压缩后的目录:
# cd redis-3.0.2
# make PREFIX=/usr/local/redis301 install // 这里说明下:要进入解压缩后的安装包目录下 进行 make 安装 ,= 号后面跟的是 要安装的目录路径。
安装完成后,可以看到/usr/local/redis301 目录下有一个 bin 目录,bin 目录里就是 redis 的命令脚本:
redis-benchmark redis-check-aof redis-check-dump redis-cli redis-server
然后分别为每个目录安装
# make PREFIX=/usr/local/redis302 install
# make PREFIX=/usr/local/redis303 install
同样的 服务器 192.168.1.105 也需要进行上面 1),2), 3), 4), 5)这五个操作步骤。不同的是步骤5)安装时,为了区分同一台服务器上安装多个redis服务,我用redis30x, 作为区分。
# make PREFIX=/usr/local/redis304 install
# make PREFIX=/usr/local/redis305 install
# make PREFIX=/usr/local/redis306 install
6)创建集群配置目录,并拷贝 redid.conf 配置文件到各节点配置目录:
192.168.1.102
# mkdir -p /usr/local/redis301/cluster/7111
# cp /usr/local/redis-3.0.2/redis.conf /usr/local/redis3/cluster/7111/redis-7111.conf
192.168.1.102
# mkdir -p /usr/local/redis302/cluster/7112
# cp /usr/local/redis-3.0.2/redis.conf /usr/local/redis302/cluster/7112/redis-7112.conf
192.168.1.102
# mkdir -p /usr/local/redis303/cluster/7113
# cp /usr/local/redis-3.0.2/redis.conf /usr/local/redis303/cluster/7113/redis-7113.conf
192.168.1.105
# mkdir -p /usr/local/redis304/cluster/7114
# cp /usr/local/redis-3.0.3/redis.conf /usr/local/redis304/cluster/7114/redis-7114.conf
192.168.1.105
# mkdir -p /usr/local/redis305/cluster/7115
# cp /usr/local/redis-3.0.2/redis.conf /usr/local/redis305/cluster/7115/redis-7115.conf
192.168.1.105
# mkdir -p /usr/local/redis306/cluster/7116
# cp /usr/local/redis-3.0.2/redis.conf /usr/local/redis306/cluster/7116/redis-7116.conf
7)拷贝完成之后要分别拷贝后的配置文件, redis-7111.config ~ redis-7116.config 六个配置文件。配置文件中的下面选项需要修改的地方如下:
配置选项 | 选项值 | 说明 |
|
|
|
daemonize | yes | 是否作为守护进程运行 |
|
|
|
pidfile | /var/run/redis-7111.pid | 如以后台进程运行,则需指定一个 pid, |
|
| 默认为/var/run/redis.pid |
|
|
|
port | 7111 | 监听端口,默认为 6379 |
|
| 注意:集群通讯端口值默认为此端口值+10000,如17111 |
|
|
|
databases | 1 | |
|
| 号 ID 库中,无特殊需求,建议仅设置一个数据库 |
|
| databases 1 |
|
|
|
cluster-enabled | yes | 打开 redis 集群 |
|
|
|
cluster-config-file | /usr/local/redis301/cluste | 集群配置文件(启动自动生成),不用人为干涉,但路径需要配置为相应的redis安装目录 |
| r/7111/nodes.conf |
|
|
|
|
cluster-node-timeout | 15000 | 节点互连超时时间。毫秒 |
|
|
|
cluster-migration- | 1 | 数据迁移的副本临界数,这个参数表示的是,一个主节 |
barrier |
| 点在拥有多少个好的从节点的时候就要割让一个从节 |
|
| 点出来给另一个没有任何从节点的主节点。 |
|
|
|
cluster-require-full- | yes | 如果某一些 key space 没有被集群中任何节点覆盖,集 |
coverage |
| 群将停止接受写入。 |
|
|
|
appendonly | yes | 启用 aof 持久化方式 |
|
| 因为 redis 本身同步数据文件是按上面 save 条件来同 |
|
| 步的,所以有的数据会在一段时间内只存在于内存中。 |
|
| 默认值为 no |
|
|
|
dir | /usr/local/redis301/cluste | 节点数据持久化存放目录(建议配置),需要注意每个redis服务器的安装目录不一样,这里只举例为redis301 |
| r/7111 |
|
|
|
|
192.168.1.102
vi /usr/local/redis301/cluster/7111/redis-7111.conf
pidfile /var/run/redis-7111.pid
port 7111
cluster-config-file /usr/local/redis301/cluster/7111/nodes.conf
dir /usr/local/redis301/cluster/7111
192.168.1.102
vi /usr/local/redis302/cluster/7112/redis-7112.conf
pidfile /var/run/redis-7112.pid
port 7112
cluster-config-file /usr/local/redis302/cluster/7112/nodes.conf
dir /usr/local/redis302/cluster/7113
192.168.1.102
vi /usr/local/redis303/cluster/7113/redis-7113.conf
pidfile /var/run/redis-7113.pid
port 7113
cluster-config-file /usr/local/redis303/cluster/7113/nodes.conf
dir /usr/local/redis303/cluster/7113
192.168.1.105
vi /usr/local/redis304/cluster/7114/redis-7114.conf
pidfile /var/run/redis-7114.pid
port 7114
cluster-config-file /usr/local/redis304/cluster/7114/nodes.conf
dir /usr/local/redis304/cluster/7114
192.168.1.105
vi /usr/local/redis305/cluster/7115/redis-7115.conf
pidfile /var/run/redis-7115.pid
port 7115
cluster-config-file /usr/local/redis305/cluster/7115/nodes.conf
dir /usr/local/redis305/cluster/7115
192.168.1.105
vi /usr/local/redis306/cluster/7116/redis-7116.conf
pidfile /var/run/redis-7116.pid
port 7116
cluster-config-file /usr/local/redis306/cluster/7116/nodes.conf
dir /usr/local/redis306/cluster/7116
包含了最少选项的集群配置文件示例如下:
port 7000 cluster-enabled yes
cluster-config-file nodes.conf cluster-node-timeout 5000
appendonly yes
使用如下命令启动这 6 个 Redis 节点实例:
192.168.1.102
# /usr/local/redis301/bin/redis-server /usr/local/redis301/cluster/7111/redis-7111.conf
192.168.1.102
# /usr/local/redis302/bin/redis-server /usr/local/redis302/cluster/7112/redis-7112.conf
192.168.1.102
# /usr/local/redis303/bin/redis-server /usr/local/redis303/cluster/7113/redis-7113.conf
192.168.1.105
# /usr/local/redis304/bin/redis-server /usr/local/redis304/cluster/7114/redis-7114.conf
192.168.1.105
# /usr/local/redis305/bin/redis-server /usr/local/redis305/cluster/7115/redis-7115.conf
192.168.1.105
# /usr/local/redis306/bin/redis-server /usr/local/redis306/cluster/7116/redis-7116.conf
启动之后用 PS 命令查看实例启动情况:
服务器 192.168.1.102 上查看 redis 进程如下:
# ps -ef | grep redis
root 5443 1 0 22:49 ? 00:00:00 /usr/local/redis301/bin/redis-server *:7111 [cluster]
root 5421 1 0 22:49 ? 00:00:00 /usr/local/redis302/bin/redis-server *:7112 [cluster]
root 5457 1 0 22:49 ? 00:00:00 /usr/local/redis303/bin/redis-server *:7113 [cluster]
服务器 192.168.1.105 上查看redis进程如下:
# ps -ef | grep redis
root 5379 1 0 22:50 ? 00:00:00 /usr/local/redis304/bin/redis-server *:7114 [cluster]
root 5331 1 0 22:50 ? 00:00:00 /usr/local/redis305/bin/redis-server *:7115 [cluster]
root 5687 1 0 22:50 ? 00:00:00 /usr/local/redis306/bin/redis-server *:7116 [cluster]
此时仅仅是每个 redis服务器启动起来了,但集群 构建成功。接下来准备创建集群
8)安装 ruby 和 rubygems(注意:需要 ruby 的版本在 1.8.7 以上)
# yum install ruby rubygems
检查 ruby 版本:
# ruby -v
ruby 2.0.0 (2017-03-24 patchlevel 374) [x86_64-Linux]
9) gem 安装 redis ruby 接口:
# gem install redis
Successfully installed redis-3.2.1
1 gem installed
Installing ri documentation for redis-3.2.1...
Installing RDoc documentation for redis-3.2.1...
10) 执行 Redis 集群创建命令(只需要在其中一个节点上执行一次则可)
# cd /usr/local/redis-3.0.2/src/ //注意这里不是安装目录 而是安装包解压缩后的 文件目录
# cp redis-trib.rb /usr/local/bin/redis-trib
# redis-trib create --replicas 1 192.168.1.102:7111192.168.1.102:7112 192.168.1.102:7113 192.168.1.105:7114 192.168.1.105:7115 192.168.1.105:7116
会产生如下log :
>>> Creating cluster
Connecting to node 192.168.1.105:7111: OK
Connecting to node 192.168.1.105:7112: OK
Connecting to node 192.168.1.105:7116: OK
Connecting to node 192.168.1.102:7113: OK
Connecting to node 192.168.1.102:7114: OK
Connecting to node 192.168.1.102:7115: OK
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.1.105:7111
192.168.1.102:7113
192.168.1.105:7112
Adding replica 192.168.1.102:7114 to 192.168.1.105:7111
Adding replica 192.168.1.105:7116 to 192.168.1.102:7113
Adding replica 192.168.1.102:7115 to 192.168.1.105:7112
M: 4f9560a0d73373f4d3cd6423361793496d79cce1 192.168.1.105:7111
slots:0-5460 (5461 slots) master
M: 8078de77fe71573256a76245b39be38f04a31a40 192.168.1.105:7112
slots:10923-16383 (5461 slots) master
S: 0b7059ff8473dbc7187be8182bf3ac9525740e35 192.168.1.105:7116
replicates 39d2eb831d9db8ceb822549378978b4db10fc3f3
M: 39d2eb831d9db8ceb822549378978b4db10fc3f3 192.168.1.102:7113
slots:5461-10922 (5462 slots) master
S: a7e98665796d8cea14aafbbebabbf88f5ac4222d 192.168.1.102:7114
replicates 4f9560a0d73373f4d3cd6423361793496d79cce1
S: 8f74a54f423ab947165bda4ff9876ceed8503850 192.168.1.102:7115
replicates 8078de77fe71573256a76245b39be38f04a31a40
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join....
>>> Performing Cluster Check (using node 192.168.1.105:7111)
M: 4f9560a0d73373f4d3cd6423361793496d79cce1 192.168.1.105:7111
slots:0-5460 (5461 slots) master
M: 8078de77fe71573256a76245b39be38f04a31a40 192.168.1.105:7112
slots:10923-16383 (5461 slots) master
M: 0b7059ff8473dbc7187be8182bf3ac9525740e35 192.168.1.105:7116
slots: (0 slots) master
replicates 39d2eb831d9db8ceb822549378978b4db10fc3f3
M: 39d2eb831d9db8ceb822549378978b4db10fc3f3 192.168.1.102:7113
slots:5461-10922 (5462 slots) master
M: a7e98665796d8cea14aafbbebabbf88f5ac4222d 192.168.1.102:7114
slots: (0 slots) master
replicates 4f9560a0d73373f4d3cd6423361793496d79cce1
M: 8f74a54f423ab947165bda4ff9876ceed8503850 192.168.1.102:7115
slots: (0 slots) master
replicates 8078de77fe71573256a76245b39be38f04a31a40
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@localhost bin]#
此时 可以看到 7111 7112 7113 三个redis服务器被做成了 master 另外三台分别为这三台的 replicates 复制的redis从服务器
这时,集群已经将这六个节点 链接起来了,互相进行了 通信。详细原理 请参照查阅reids 官网或者 《redis设计与实现》
11)集群简单测试 :
任意 进入一个redis 安装节点
192.168.1.102 服务器:
#cd /usr/local/redis301/bin
# ./redis-cli -c -h192.168.1.102 -p 7111 //这里需要说明下 -c 参数 代表,以集群的方式登陆 7111 服务器的client 端进行操作。
192.168.1.102:7111> set test 123
OK
192.168.1.105服务器:
#cd /usr/local/redis304/bin
# ./redis-cli -c -h 192.168.1.105 -p 7114
192.168.1.105:7114> get test
'123'
至此,说明集群已经安装配置完毕。