redis部署（3台集群）自动安装

注：

• 在配置时，本文档以【用户：redis/redis】、【目录：/redis】为例进行说明
• 文档中执行命令为蓝色字体；配置文件为粉色字体；重点标记为红色字体

1. 集群部署

• redis集群为主从集群，集群服务器需要三台以上，本文档以三台进行说明
• 在部署时，每个主节点会自动生成一个从节点，比如192.168.1.1会启动一个主节点，同时会在192.168.1.2启动一个从节点
• 主节点数据可读写，从节点数据只可读，数据写入时会自动选择一个主节点存储数据，同时数据同步至从节点
• 当主节点挂掉时，从节点在10s内会自动升级为主节点，原来的主节点恢复后会自动变为从节点
• 应用连接时，需要配置三个主节点的IP和端口
  1. 架构图

1. 1. 准备
      1. 人工确认
         1. 关闭防火墙

临时停止防火墙：service iptables stop

永久停止防火墙：chkconfig iptables off

1. 1. 1. 1. 关闭SELinux

临时关闭SELinux：setenforce 0

永久关闭SELinux：vi etc/selinux/config  修改SELINUX=disabled

1. 1. 1. 1. 确认sshd服务配置正常

此步骤建议先不做，如果出现安装问题再操作

• 设置/etc/ssh/sshd_config

1. 将Protocol设置为2
2. 将AuthorizedKeysFile注释掉
3. 将PermitRootLogin设置为yes，允许root用户登录
4. 修改后重新启动sshd服务：service sshd restart

 

• 设置/etc/ssh/ssh_config

增加StrictHostKeyChecking no配置，防止主机IP变动引起的ssh登陆异常

Host *

StrictHostKeyChecking no

1. 1. 1. 1. 确认redis用户的.bash_profile文件内容

cat /redis/.bash_profile ，必须存在以下内容

-----------------------------------------------------------------------

if [ -f ~/.bashrc ]; then

        . ~/.bashrc

fi

-----------------------------------------------------------------------

1. 1. 1. 安装依赖包

此部分必须使用root用户或sodu权限用户安装

1. 1. 1. 1. 上传prepare.tar

在任意一台，上传prepare.tar至/tmp目录，解压tar xvf prepare.tar得到prepare目录

1. 1. 1. 1. 上传本地源压缩包

先确认linux系统版本：lsb_release -a 或cat /etc/redhat-release 

在REPO文件夹找到对应linux版本的本地源压缩包，上传到prepare目录

操作系统	本地源压缩包
CentOS5/Redhat5	RHEL5_REPO.tar.gz
CentOS6/Redhat6/CGSL4	RHEL6_REPO.tar.gz
CentOS7/Redhat7	RHEL7_REPO.tar.gz
openSuSE11	SUSE11_REPO.tar.gz
openSuSE12	SUSE12_REPO.tar.gz

1. 1. 1. 1. 安装

进入prepare目录，执行./preinstall.sh 'IP1, IP2 , IP3' root 'root密码'，将会安装依赖包到3台服务器

例：./preinstall.sh '192.168.1.1,192.168.1.2,192.168.1.3' root 'root123'

 

执行完之后出现以下内容证明依赖包安装成功

---------------------------------------------------------------------------------------------------------------------------------

2017-03-22 16:00:24  *  Begin to prepare for installing on [192.168.1.1,192.168.1.2,192.168.1.3]

2017-03-22 16:00:24  *  Install python on the localhost...

<<<<<<<<<<<<<< python installed yet. <<<<<<<<<<<<<<<<

 

[Preparation Tool For Installing]

Starting preparation tasks...

Hosts to prepare: ['192.168.1.1', '192.168.1.2', '192.168.1.3']

192.168.1.2: ['RHEL6']

192.168.1.3: ['RHEL6']

192.168.1.1: ['RHEL6']

 

scp /tmp/prepare/RHEL6_REPO.tar.gz to all hosts...

scp /tmp/prepare/yum-install.sh to all hosts...

exec /tmp/yum-install.sh on all hosts[repo: file:///tmp/zcache-repo]...

192.168.1.2: <<<<<<<<<<<<<< python installed yet. <<<<<<<<<<<<<<<<

192.168.1.2: >>>>>>>>>>>>>> Installing the ruby >>>>>>>>>>>>>>>>

192.168.1.2: >>>>>>>>>>>>>> Installing the rubygems >>>>>>>>>>>>>>>>

192.168.1.2: <<<<<<<<<<<<<< lua installed yet. <<<<<<<<<<<<<<<<

192.168.1.2: >>>>>>>>>>>>>> Installing the tcl >>>>>>>>>>>>>>>>

 

192.168.1.3: <<<<<<<<<<<<<< python installed yet. <<<<<<<<<<<<<<<<

192.168.1.3: >>>>>>>>>>>>>> Installing the ruby >>>>>>>>>>>>>>>>

192.168.1.3: >>>>>>>>>>>>>> Installing the rubygems >>>>>>>>>>>>>>>>

192.168.1.3: <<<<<<<<<<<<<< lua installed yet. <<<<<<<<<<<<<<<<

192.168.1.3: >>>>>>>>>>>>>> Installing the tcl >>>>>>>>>>>>>>>>

 

192.168.1.1: <<<<<<<<<<<<<< python installed yet. <<<<<<<<<<<<<<<<

192.168.1.1: >>>>>>>>>>>>>> Installing the ruby >>>>>>>>>>>>>>>>

192.168.1.1: >>>>>>>>>>>>>> Installing the rubygems >>>>>>>>>>>>>>>>

192.168.1.1: <<<<<<<<<<<<<< lua installed yet. <<<<<<<<<<<<<<<<

192.168.1.1: >>>>>>>>>>>>>> Installing the tcl >>>>>>>>>>>>>>>>

 

scp /tmp/prepare/redis.gem to all hosts...

Install /tmp/redis.gem on all hosts...

End preparation: ok

EXIT...

 

2017-03-22 16:01:45  *  Preparing done

---------------------------------------------------------------------------------------------------------------------------------

1. 1. 1. 1. 检查

3台服务器分别切换到redis用户，执行gem list，需要出现以下内容

如果出现以下报错，使用root执行chmod 755 /usr/lib/ruby

1. 1. 安装redis集群

redis可在相同机器安装多套，必须用户、用户归属组、端口、路径不同

redis安装目录如果是软连接的目录，必须把源目录的权限设置为755

1. 1. 1. 文件准备

从本章节开始全部使用redis用户操作

在任意一台机器上传rcinstall.tar至/redis目录

tar xvf rcinstall.tar

1. 1. 1. 配置

cd /redis/rcinstall

vi install.options

---------------------------------------------------------------------------------------------------------------------------------

hosts = 192.168.1.1,192.168.1.2,192.168.1.3         #redis集群安装的IP地址

install_dir = /redis                               #redis安装目录

data_dir = /redis/data                            #redis数据存放目录

redis_cluster_group = redis                        #安装redis用户的归属组

redis_cluster_masters = 3                         #见参数说明

redis_cluster_port = 7380                         #见参数说明

redis_cluster_replicas = 1                         #见参数说明

redis_cluster_user = redis                         #安装redis用户名

redis_cluster_user_password = redis                #安装redis用户密码

root_name = redis                               #安装redis用户名

root_password = redis                            #安装redis用户密码

---------------------------------------------------------------------------------------------------------------------------------

配置参数说明：

• redis_cluster_masters：集群拥有的master节点个数

master个数需要 >= 3，因为redis集群是通过选举机制来判断节点可用性，需要有半数以上master节点投票才有效。

【如果master个数小于3，那么将无法有效认定节点的可用性，当某个节点故障时，redis集群也就无法在master故障时自动进行主从切换】

 

• redis_cluster_port：redis节点开始分配的端口号

分配端口号段的开始值。例如redis_cluster_port= 7380，当主机A上部署2个redis节点，则三个节点使用的端口号将依次为7380，7381；（因为redis cluster的实现特性，redis节点同时将分别占用17380，17381端口）

 

• redis_cluster_replicas：集群中每个节点的副本个数

集群创建的节点个数为： redis_cluster_masters ×（1 + redis_cluster_replicas）

1. 1. 1. 安装

cd /redis/rcinstall

./rcinstall.py --silent=install.options

 

执行完之后出现以下内容证明安装成功，并且所有的redis都已经启动

---------------------------------------------------------------------------------------------------------------------------------

Redis Cluster Installation Tool

 

Starting installation tasks...

 

Begin Installation

['192.168.1.1', '192.168.1.2', '192.168.1.3']

Installing/Repairing SSH keys for redis

 

Creating redis Data Directory...

 

Install on all nodes...

 

Creating redis nodes: 6 = 3(masters) * [1 + 1(replicas)]

Init redis node...

 

Init redis nodes(192.168.1.1:7380) ok.

Init redis nodes(192.168.1.2:7380) ok.

Init redis nodes(192.168.1.3:7380) ok.

Init redis nodes(192.168.1.1:7381) ok.

Init redis nodes(192.168.1.2:7381) ok.

Init redis nodes(192.168.1.3:7381) ok.

>>> Creating cluster

Connecting to node 192.168.1.1:7380: OK

Connecting to node 192.168.1.2:7380: OK

Connecting to node 192.168.1.3:7380: OK

Connecting to node 192.168.1.1:7381: OK

Connecting to node 192.168.1.2:7381: OK

Connecting to node 192.168.1.3:7381: OK

>>> Performing hash slots allocation on 6 nodes...

Using 3 masters:

192.168.1.1:7380

192.168.1.2:7380

192.168.1.3:7380

Adding replica 192.168.1.2:7381 to 192.168.1.1:7380

Adding replica 192.168.1.3:7381 to 192.168.1.2:7380

Adding replica 192.168.1.1:7381 to 192.168.1.3:7380

M: 83e4b0f93c179372f6fc1ef698214d3ede6cb3de 192.168.1.1:7380

   slots:0-5460 (5461 slots) master

M: cc2a7c1836d1a6109e43fbdbcba1d94590d789df 192.168.1.2:7380

   slots:5461-10922 (5462 slots) master

M: a8f1cdfd6d0243675e5793f93d8e841268354327 192.168.1.3:7380

   slots:10923-16383 (5461 slots) master

S: f6fb2e2bfdbf90400639b8e84eba09908d340c4c 192.168.1.1:7381

   replicates a8f1cdfd6d0243675e5793f93d8e841268354327

S: a8194f5dc8ddab96c2a5dded5bb504036966a714 192.168.1.2:7381

   replicates 83e4b0f93c179372f6fc1ef698214d3ede6cb3de

S: c2b6531d694a69fe34de4c8813bb081768c2e2b1 192.168.1.3:7381

   replicates cc2a7c1836d1a6109e43fbdbcba1d94590d789df

>>> Nodes configuration updated

>>> Assign a different config epoch to each node

>>> Sending CLUSTER MEET messages to join the cluster

Waiting for the cluster to join...

>>> Performing Cluster Check (using node 192.168.1.1:7380)

M: 83e4b0f93c179372f6fc1ef698214d3ede6cb3de 192.168.1.1:7380

   slots:0-5460 (5461 slots) master

M: cc2a7c1836d1a6109e43fbdbcba1d94590d789df 192.168.1.2:7380

   slots:5461-10922 (5462 slots) master

M: a8f1cdfd6d0243675e5793f93d8e841268354327 192.168.1.3:7380

   slots:10923-16383 (5461 slots) master

M: f6fb2e2bfdbf90400639b8e84eba09908d340c4c 192.168.1.1:7381

   slots: (0 slots) master

   replicates a8f1cdfd6d0243675e5793f93d8e841268354327

M: a8194f5dc8ddab96c2a5dded5bb504036966a714 192.168.1.2:7381

   slots: (0 slots) master

   replicates 83e4b0f93c179372f6fc1ef698214d3ede6cb3de

M: c2b6531d694a69fe34de4c8813bb081768c2e2b1 192.168.1.3:7381

   slots: (0 slots) master

   replicates cc2a7c1836d1a6109e43fbdbcba1d94590d789df

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

---------------------------------------------------------------------------------------------------------------------------------

1. 1. 1. 测试

进入任意一台的/redis/redis/bin

cd /redis/redis/bin

./redis-cli -h 192.168.1.1 -p 7380 -c

---------------------------------------------------------------------------------------------------------------------------------

crm89:/redis/redis/bin $ ./redis-cli -h 192.168.1.1 -p 7380 -c

192.168.1.1:7380> set key 1

-> Redirected to slot [12539] located at 192.168.1.3:7380

OK

192.168.1.3:7380> get key

"1"

192.168.1.3:7380> exit

---------------------------------------------------------------------------------------------------------------------------------

1. 1. 1. 启动

进入任意一台的/redis/redis/bin

cd /redis/redis/bin

./rc-start.sh

1. 1. 1. 停止

进入任意一台的/redis/redis/bin

cd /redis/redis/bin

./rc-stop.sh

• 1. 1. 更改缓存类型
• 方法1：

redis安装时，默认同时写缓存和硬盘，缓存和硬盘的数据保持一致性，如果只需要做内存缓存时，需要修改以下位置，修改完之后需要重启：

进入安装目录的conf目录，修改所有redis-端口.conf

---------------------------------------------------------------------------------------------------------------------------------

#appendonly yes                  #注释掉

appendonly no                  #增加此行

# appendfsync always

#appendfsync everysec             #注释掉

 appendfsync no                  #放开

# no-appendfsync-on-rewrite yes

auto-aof-rewrite-percentage 200

auto-aof-rewrite-min-size 256mb

---------------------------------------------------------------------------------------------------------------------------------

 

• 方法2：

或者登录每个节点关闭持久化，执行更改命令，不需要重启

./redis-cli -h 192.168.1.1 -p 7380   #注意：每台机器的所有redis节点都需登录执行

config set appendonly no         #关闭持久化

config rewrite                  #写入配置文件

1. 1. 1. 重新安装

在安装中可能出错，需要重新安装redis，以下操作每台机器都需要操作

1. 使用ps -ef|grep 查看redis的进程，并kill所有进程
2. 删除原安装目录的文件，如：原配置的【install_dir = /redis 】，则删除/redis/redis
3. 重新进行安装
   1. 1. 节点信息

执行以下命令可以查看到节点信息：

./redis-cli -h 192.168.1.1 -p 7380 cluster nodes

如果发现一台机器有两个从节点，需要升级一个从节点为主节点（默认为7380为节点端口），否则会出现单节点故障：

./redis-cli -h 192.168.1.1 -p 7380 cluster failover