redis-cluster分布式集群安装部署

一.服务器环境

1.1环境说明

此次演示是3台内网机器，开启6个redis单实例。其中每台机器上按照不同的端redis服务端口开启2个redis实例

环境：机器的内网IP地址：

172.168.1.101
172.168.1.103
172.168.1.104

3台服务器事前先关闭iptables ,开启时间同步，关闭selinux.
服务器的系统环境是centos6.5 X86_64位的 最小化安装

1.2安装redis服务,开启redis实例

以172.168.1.101机器为例来安装redis以及开启2个不同的redis实例

tar xf redis-4.0.6.tar.gz
cd redis-4.0.6
make MANIFESTO=jemalloc
make PREFIX=/usr/local/redis-4.0.6 install
ln -s /usr/local/redis-4.0.6/ /usr/local/redis
echo "export PATH=/usr/local/redis/bin:$PATH"  >>/etc/profile
source /etc/profile
mkdir  /usr/local/redis/conf/
cp /root/redis-4.0.6/redis.conf /usr/local/redis/conf/
mkdir -p  /data/redis/{10681,21683}
cp /usr/local/redis/conf/redis.conf /usr/local/redis/conf/10681.conf
cp /usr/local/redis/conf/redis.conf /usr/local/redis/conf/21683.conf

准备redis配置文件：
[root@master conf]# tail -11 /usr/local/redis/conf/10681.conf

# active-defrag-cycle-max 75
pidfile /var/run/redis10681.pid
logfile "/data/redis/10681/10681.log"
bind 172.168.1.101
daemonize yes 
dir   /data/redis/10681
#requirepass  DJOEr=+590dLe
cluster-enabled yes 
cluster-config-file nodes-10681.conf
#cluster-node-timeout 15000

[root@master conf]# tail -11 /usr/local/redis/conf/21683.conf

#active-defrag-cycle-max 75

pidfile /var/run/redis21683.pid
logfile "/data/redis/21683/21683.log"
bind 172.168.1.101
daemonize yes 
dir   /data/redis/21683
#requirepass  DJOEr=+590dLe
cluster-enabled yes 
cluster-config-file nodes-21683conf
#cluster-node-timeout 15000

二、部署Ruby环境

/opt/redis-3.2.8/src/redis-trib.rb 这个命令是基于Ruby域名编写的，所以需要有Ruby环境才可以运行,所以要准备redis的Ruby环境。

第一种方法

1.安装Ruby环境

yum -y install ruby ruby-devel
yum -y install rubygems

2.添加淘宝的Ruby

[root@git-server ~]#gem sources --remove http://rubygems.org/   http://rubygems.org/ removed from sources

 添加淘宝ruby源;
[root@git-server ~]#gem sources --add https://ruby.taobao.org/   https://ruby.taobao.org/ added to sources
查看现有ruby源;
[root@git-server ~]#gem sources -l
*** CURRENT SOURCES ***
https://ruby.taobao.org/

3.安装curl和RVM

[root@git-server ~]#sudo yum install curl -y

安装RVM

[root@git-server ~]#curl -L get.rvm.io | bash -s stable
[root@git-server ~]#source/usr/local/rvm/scripts/rvm

4.查看rvm库中已知的ruby版本

[root@git-server ~]#rvm list known   
MRI Rubies
[ruby-]2.0.0[-p648]
[ruby-]2.1[.10]
[ruby-]2.2[.7]
[ruby-]2.3[.4]
[ruby-]2.4[.1]
**安装一个ruby版本**
[root@git-server ~]#rvm install 2.3.4  #### **这里要选择正确 **

[root@git-server ~]# ruby --version
ruby 1.8.7 (2013-06-27 patchlevel 374) [x86_64-linux]

[root@git-server ~]# rvm --create ruby-2.3.4
Warning! PATH is not properly set up, '/usr/local/rvm/gems/ruby-2.3.4/bin' is not available.
         Usually this is caused by shell initialization files. Search for 'PATH=...' entries.
         You can also re-add RVM to your profile by running: 'rvm get stable --auto-dotfiles'.
         To fix it temporarily in this shell session run: 'rvm use ruby-2.3.4'.
         To ignore this error add rvm_silence_path_mismatch_check_flag=1 to your ~/.rvmrc file.
Using /usr/local/rvm/gems/ruby-2.3.4

5、查看当前的Ruby的版本

[root@git-server ~]# 
[root@git-server ~]# ruby --version
ruby 2.3.4p301 (2017-03-30 revision 58214) [x86_64-linux]
[root@git-server ~]# gem install redis --version 3.2.8
ERROR:  Could not find a valid gem 'redis' (= 3.2.8) in any repository

6、安装报错，于是换个redis的版本，安装成功

[root@git-server ~]# gem install redis --version 3.2.1
Fetching: redis-3.2.1.gem (100%)
Successfully installed redis-3.2.1
Parsing documentation for redis-3.2.1
Installing ri documentation for redis-3.2.1
Done installing documentation for redis after 2 seconds
1 gem installed

redis-trib.rb 出现如下帮助信息说明安装成功

[root@git-server ~]# /opt/redis-3.2.8/src/redis-trib.rb
Usage: redis-trib <command> <options> <arguments ...>

  create          host1:port1 ... hostN:portN
                  --replicas <arg>
  check           host:port
  info            host:port
  fix             host:port
                  --timeout <arg>
  reshard         host:port
                  --from <arg>
                  --to <arg>
                  --slots <arg>
                  --yes
                  --timeout <arg>
                  --pipeline <arg>
  rebalance       host:port
                  --weight <arg>
                  --auto-weights
                  --use-empty-masters
                  --timeout <arg>
                  --simulate
                  --pipeline <arg>
                  --threshold <arg>
  add-node        new_host:new_port existing_host:existing_port
                  --slave
                  --master-id <arg>
  del-node        host:port node_id
  set-timeout     host:port milliseconds
  call            host:port command arg arg .. arg
  import          host:port
                  --from <arg>
                  --copy
                  --replace
  help            (show this help)

For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.
[root@git-server ~]# 

提示第一种方法安装的Ruby环境，每次执行redis-trib.rb命令必须先在命令行加载 Ruby环境才可以
[root@git-server ~]#source/usr/local/rvm/scripts/rvm

第二种方法：

yum install rubygems  ruby ruby-devel -y
gem install redis --version 3.2.1

[root@git-server src]# which gem
/usr/bin/gem
[root@git-server src]# gem install redis --version 3.2.1
Successfully installed redis-3.2.1
1 gem installed
Installing ri documentation for redis-3.2.1...
Installing RDoc documentation for redis-3.2.1..

如果连接不上gem服务器安装，就手动下载并安装

gem install redis --version 3.0.0  
ERROR:  Could not find a valid gem 'redis' (= 3.0.0) in any repository
ERROR:  While executing gem ... (Gem::RemoteFetcher::FetchError)

需要手工下载并安装：

wget https://rubygems.global.ssl.fastly.net/gems/redis-3.2.1.gem
gem install -l ./redis-3.2.1.gem

[root@git-server src]# cp redis-trib.rb /usr/local/bin/
[root@git-server src]# which redis-trib.rb
/usr/local/bin/redis-trib.rb

三、开启redis-cluster集群

将6台独立的redis实例加入到redis集群中
此处的--replicas 1 1代表是6个redis实例组建成一个3主3从的redis的分部署集群

[root@git-server src]# ./redis-trib.rb create --replicas 1 172.168.1.104:10681 172.168.1.103:10681 172.168.1.101:10681 172.168.1.103:4683 172.168.1.101:4683172.168.1.104:4683

>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
172.168.1.101:10681
172.168.1.103:10681
172.168.1.104:10681
Adding replica 172.168.1.103:4683 to 172.168.1.101:10681
Adding replica 172.168.1.101:4683 to 172.168.1.103:10681
Adding replica 172.168.1.104:4683 to 172.168.1.104:10681
M: f854b48f429ba7367ec03a34a8cd687e826f37b3 172.168.1.104:10681
   slots:10923-16383 (5461 slots) master
M: 640a40e72f0a0e8bb9362bc625aae843d02104e3 172.168.1.103:10681
   slots:5461-10922 (5462 slots) master
M: 2b637c94f6bd34928237c6c30257e283de36aa50 172.168.1.101:10681
   slots:0-5460 (5461 slots) master
S: 7da1166eda6e8e80febe37b2294c65c875d5790b 172.168.1.103:4683
   replicates 2b637c94f6bd34928237c6c30257e283de36aa50
S: 558a242dd07d265db3163fad12484433df97c240 172.168.1.101:4683
   replicates 640a40e72f0a0e8bb9362bc625aae843d02104e3
S: 408820aa4ae10accbc4c06945f802c1b47a12bd4 172.168.1.104:4683
   replicates f854b48f429ba7367ec03a34a8cd687e826f37b3
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join..
>>> Performing Cluster Check (using node 172.168.1.104:10681)
M: f854b48f429ba7367ec03a34a8cd687e826f37b3 172.168.1.104:10681
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
M: 640a40e72f0a0e8bb9362bc625aae843d02104e3 172.168.1.103:10681
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: 408820aa4ae10accbc4c06945f802c1b47a12bd4 172.168.1.104:4683
   slots: (0 slots) slave
   replicates f854b48f429ba7367ec03a34a8cd687e826f37b3
S: 558a242dd07d265db3163fad12484433df97c240 172.168.1.101:4683
   slots: (0 slots) slave
   replicates 640a40e72f0a0e8bb9362bc625aae843d02104e3
S: 7da1166eda6e8e80febe37b2294c65c875d5790b 172.168.1.103:4683
   slots: (0 slots) slave
   replicates 2b637c94f6bd34928237c6c30257e283de36aa50
M: 2b637c94f6bd34928237c6c30257e283de36aa50 172.168.1.101:10681
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

登录一个节点 查看redis节点的slot的分配情况

[root@slave02 src]# redis-cli -h 172.168.1.104 -p 10681 -c
172.168.1.104:10681> cluster nodes
640a40e72f0a0e8bb9362bc625aae843d02104e3 172.168.1.103:10681@20681 master - 0 1529883076949 2 connected 5461-10922
f854b48f429ba7367ec03a34a8cd687e826f37b3 172.168.1.104:10681@20681 myself,master - 0 1529883075000 1 connected 10923-16383
408820aa4ae10accbc4c06945f802c1b47a12bd4 172.168.1.104:4683@14683 slave f854b48f429ba7367ec03a34a8cd687e826f37b3 0 1529883078984 6 connected
558a242dd07d265db3163fad12484433df97c240 172.168.1.101:4683@14683 slave 640a40e72f0a0e8bb9362bc625aae843d02104e3 0 1529883077971 5 connected
7da1166eda6e8e80febe37b2294c65c875d5790b 172.168.1.103:4683@14683 slave 2b637c94f6bd34928237c6c30257e283de36aa50 0 1529883077000 4 connected
2b637c94f6bd34928237c6c30257e283de36aa50 172.168.1.101:10681@20681 master - 0 1529883078000 3 connected 0-5460
172.168.1.104:10681> 

到此处集群创建完成
提示：
要求一开始不要配置cluster集群，
注意一：！！！直接开启6个redis实例便可以，千万不要画蛇添足，一开始就给给配置成3组redis的主从复制实例

注意二：可以集群中的每个redis实例配置密码.另外，由于下面我们需要配置集群密码，故之前配置的 requirepass 先删掉，集群配置成功后再对6个实例进行配置密码.

注意三：在给6个redis组建集群时，同一机器上的redis实例不要分配主从到同一台机器上，也就是同一机器上的2个实例要和其他的机器上的redis实例交替开
类似于这样的组合：
172.168.1.104:10681 172.168.1.103:10681 172.168.1.101:10681 172.168.1.103:4683 172.168.1.101:4683 172.168.1.104:4683
参考博文：
https://blog.csdn.net/localhost01/article/details/71436801
http://www.656463.com/redis/ZZFVNb.htm

转载于:https://blog.51cto.com/wujianwei/2136271