Linux下搭建Nginx+Keepalived+Redis集群+Tomcat实现高可用WEB负载均衡、session共享集群（二）

接上一篇环境安装请参考另一篇文章

5、相关软件配置

为了测试这里将服务器防火墙直接关闭（以centos 6.5关闭防火墙为例）

service iptables stop

5.1 NGINX配置

（1）修改安装路径下conf/nginx.conf文件

vim /home/nginx/conf/nginx.conf

####负载均衡配置参考如下###

worker_processes  1;

events {
        worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    upstream platform{
        server 192.168.10.13:8080 weight=10;#填写准备的用于集群的交换平台的地址
        server 192.168.10.14:8080 weight=10;
	}
    server {
        listen       80;
        server_name  localhost;
        location ^~/platform/ {
            proxy_connect_timeout 300;
            proxy_send_timeout 300;
            proxy_read_timeout 300;
            proxy_next_upstream http_502 http_504 http_404 error timeout invalid_header;
            proxy_set_header Host $host;
            proxy_set_header X-Real-Ip $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_http_version 1.1;
            proxy_pass http://platform;
        }
    }
}

（2）重启NGINX

/home/nginx/sbin/nginx -s reload

5.2 Keepalived配置

（1）修改主机配置

vim /etc/keepalived/keepalived.conf

! Configuration File for keepalived
#邮件配置，不需要可以忽略
global_defs {
   router_id front166 #主机名
}
#nginx探测脚本
vrrp_script check_nginx {
    script "/ect/keepalived/chech_nginx.sh"
    interval 3 #执行周期，3秒一次
    weight -20 #权重
  }
#核心配置
vrrp_instance VI_1 {
    state MASTER #主机配置MASTER，备机配置BACKUP
    interface eth0 #网卡名称，以本机在用网卡为准
    virtual_router_id 51	#此id 主备相同
    priority 100  #权重，主机值要大于备机值，主备间抢夺虚拟IP
    advert_int 1
    authentication {
        auth_type PASS #主备间的验证方式，密码验证
        auth_pass 1111 #主备相同
    }
    virtual_ipaddress {
        192.168.10.16 #虚拟IP，亦VIP
    }
	track_script {	#如果不需要监控请注释此部分
		check_nginx #追踪脚本执行
   }
}

（2）新增nginx监控脚本

vim /etc/keepalived/chech_nginx.sh

#!/bin/bash
A=`ps -C nginx --no-header |wc -l`
if [ $A -eq 0 ]; then
	/home/nginx/sbin/nginx
	sleep 3
	if [ `ps -C nginx --no-header |wc -l` -eq 0 ]; then
		killall keepalived
	fi
fi

（3）重启Keepalived

service keepalived restart

（4）添加防火墙策略

firewall-cmd --direct --permanent --add-rule ipv4 filter INPUT 0 --in-interface ens33 --destination 224.0.0.18 --protocol vrrp -j ACCEPT
firewall-cmd --direct --permanent --add-rule ipv4 filter OUTPUT 0 --out-interface ens33 --destination 224.0.0.18 --protocol vrrp -j ACCEPT
firewall-cmd --reload

（5）查看虚拟IP是否创建成功

使用命令ifconfig查看是否成功，成功截图如下

ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:74:d4:df brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.13/24 brd 192.168.10.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 192.168.10.16/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::736:7d6c:2f65:7015/64 scope link noprefixroute
       valid_lft forever preferred_lft forever

若未创建成功可以手动添加VIP

ifconfig eth0:1 192.168.10.16 netmask 255.255.255.0 up

5.3 Redis集群配置

此部分参考Redis集群环境搭建（基于Redis-5.0.9版本）
这里两个虚拟机上分别创建三个redis节点（即三主三从）

（1）进入安装路径下，创建 Redis 子节点

cd /home/redis
mkdir cluster7001 cluster7002 cluster7003
cp redis.conf cluster7001/redis7001.conf
cp redis.conf cluster7001/redis7002.conf
cp redis.conf cluster7001/redis7003.conf
cp bin/redis-server cluster7001/
cp bin/redis-server cluster7002/
cp bin/redis-server cluster7003/

（2）分别修改配置文件

分别修改这三个配置文件，修改如下内容

port 7001 //端口7001,7002,7003 
bind 本机ip //默认ip为127.0.0.1 需要改为其他节点机器可访问的ip 否则创建集群时无法访问对应的端口，无法创建集群 
daemonize yes //redis后台运行 
pidfile /home/run/redis_7001.pid //pidfile文件对应7001,7002,7003 
logfile "/var/log/redis.log" 	#配置log文件地址，默认使用标准输出，即打印在命令行终端的端口上
cluster-enabled yes //开启集群 把注释#去掉 
cluster-config-file nodes_7001.conf //集群的配置 配置文件首次启动自动生成 7000,7001,7002 把注释#去掉 
cluster-node-timeout 15000 //请求超时 默认15秒，可自行设置  把注释#去掉 
appendonly yes //aof日志开启 有需要就开启，它会每次写操作都记录一条日志

（3）启动各个节点

cd /home/redis/cluster7001/
./redis-server redis7001.conf 
cd /home/redis/cluster7002/
./redis-server redis7002.conf 
cd /home/redis/cluster7003/
./redis-server redis7003.conf 

（4）创建集群

原命令 redis-trib.rb 工具目前已经废弃，5.0.*版本以后使用redis-cli进行集群创建

cd /home/redis/
./bin/redis-cli --cluster create --cluster-replicas 1  192.168.10.13:7001 192.168.10.13:7002 192.168.10.13:7003 192.168.10.14:7001 192.168.10.14:7002 192.168.10.14:7003 

Performing hash slots allocation on 6 nodes…
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.10.14:7002 to 192.168.10.13:7003
Adding replica 192.168.10.13:7002 to 192.168.10.14:7003
Adding replica 192.168.10.14:7001 to 192.168.10.13:7001
M: 055171c6821f773382fed79f627ecd1eb372bf2d 192.168.10.13:7003
slots:[0-5460] (5461 slots) master
M: 4d01aa882e1cde20c56b77724460b0a808cc0924 192.168.10.13:7001
slots:[10923-16383] (5461 slots) master
S: 24549fb047964a869fb3238bffdd664cf61eca8f 192.168.10.13:7002
replicates a55b11995743ebe67935267786a364231fc453b4
M: a55b11995743ebe67935267786a364231fc453b4 192.168.10.14:7003
slots:[5461-10922] (5462 slots) master
S: 5250e345178249a2af3a98ab14a670edca340177 192.168.10.14:7001
replicates 4d01aa882e1cde20c56b77724460b0a808cc0924
S: 9fbd454ee662b5ec3bffc9bbf3e08060bf30a07b 192.168.10.14:7002
replicates 055171c6821f773382fed79f627ecd1eb372bf2d
Can I set the above configuration? (type ‘yes’ to accept): yes

输入 yes 即可，然后出现如下内容，说明安装成功

Nodes configuration updated
Assign a different config epoch to each node
Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
…
Performing Cluster Check (using node 192.168.10.13:7003)
M: 055171c6821f773382fed79f627ecd1eb372bf2d 192.168.10.13:7003
slots:[0-5460] (5461 slots) master
1 additional replica(s)
S: 24549fb047964a869fb3238bffdd664cf61eca8f 192.168.10.13:7002
slots: (0 slots) slave
replicates a55b11995743ebe67935267786a364231fc453b4
S: 9fbd454ee662b5ec3bffc9bbf3e08060bf30a07b 192.168.10.14:7002
slots: (0 slots) slave
replicates 055171c6821f773382fed79f627ecd1eb372bf2d
M: a55b11995743ebe67935267786a364231fc453b4 192.168.10.14:7003
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
M: 4d01aa882e1cde20c56b77724460b0a808cc0924 192.168.10.13:7001
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: 5250e345178249a2af3a98ab14a670edca340177 192.168.10.14:7001
slots: (0 slots) slave
replicates 4d01aa882e1cde20c56b77724460b0a808cc0924
[OK] All nodes agree about slots configuration.
Check for open slots…
Check slots coverage…
[OK] All 16384 slots covered.

（4）集群验证

在第一台机器上连接集群的7001端口的节点，在另外一台连接7003节点，连接方式为

cd /home/redis/
./bin/redis-cli -h 192.168.10.13 -c -p 7001

在7001节点执行命令 set test 123 ，执行结果如下：

192.168.10.13:7001> set test 123
-> Redirected to slot [6918] located at 192.168.10.14:7001
OK

然后在另外一台7003端口，查看 key 为 test 的内容， get test ，执行结果如下：

192.168.10.14:7003> get test
"123"

说明集群运作正常。

5.4、tomcat配置

（1）上传用于同步session的相关redis jar包

将 jedis.jar、commons-pool2.jar、slf4j-api.jar、tomcat-cluster-redis-session-manager-3.0.5.1.jar移动到tomcat/lib/下（非项目的lib下）

（2）在tomcat/conf下创建redis-data-cache.properties文件

#并添加如下内容
redis.hosts=192.168.10.13:7001  #集群主节点
redis.cluster.enabled=true
redis.sentinel.enabled=false
redis.sentinel.master=mymaster
lb.sticky-session.enabled=false
session.persistent.policies=DEFAULT
redis.sso.timeout=0

（3）在 tomcat/conf/context.xml 文件中添加如下内容

<Valve className="tomcat.request.session.redis.SessionHandlerValve" />
<Manager className="tomcat.request.session.redis.SessionManager" />

（4）启动tomcat

cd  tomcat/bin/
./catlina.sh run

至此安装配置就已经完成了，验证我这里就不贴上来了