Keepalived+Nginx实现高可用负载均衡集群

 

一 环境介绍

1.操作系统
CentOS Linux release 7.2.1511 (Core)

2.服务
keepalived+nginx双主高可用负载均衡集群及LAMP应用
keepalived-1.2.13-7.el7.x86_64
nginx-1.10.2-1.el7.x86_64
httpd-2.4.6-45.el7.centos.x86_64


二 原理及拓扑图

1.vrrp协议
在现实的网络环境中,两台需要通信的主机大多数情况下并没有直接的物理连接。对于这样的情况,它们之间路由怎样选择?主机如何选定到达目的主机的下一跳路由,这个问题通常的解决方法有二种:
 在主机上使用动态路由协议(RIP、OSPF等) 
 在主机上配置静态路由 
很明显,在主机上配置动态路由是非常不切实际的,因为管理、维护成本以及是否支持等诸多问题。配置静态路由就变得十分流行,但路由器(或者说默认网关default gateway)却经常成为单点故障。VRRP的目的就是为了解决静态路由单点故障问题,VRRP通过一竞选(election)协议来动态的将路由任务交给LAN中虚拟路由器中的某台VRRP路由器。

2.nginx反代
nginx是以反向代理的方式进行负载均衡的。反向代理(Reverse Proxy)方式是指以代理服务器来接受Internet上的连接请求,然后将请求转发给内部网络上的服务器,并将从服务器上得到的结果返回给Internet上请求连接的客户端,此时代理服务器对外就表现为一个服务器。(为了理解反向代理,这里插播一条什么是正向代理:正向代理指的是,一个位于客户端和原始服务器之间的服务器,为了从原始服务器取得内容,客户端向代理发送一个请求并指定目标(原始服务器),然后代理向原始服务器转交请求并将获得的内容返回给客户端。)
3.拓扑图


三 配置

1.后端RS配置

1
2
[root@inode4 ~] # yum install httpd -y
[root@inode5 ~] # yum install httpd -y

 

 

2.Nginx反代配置
MASTER:

1
2
3
4
5
6
7
8
9
10
upstream websrvs {
  server 172.18.67.11:80;
  server 172.18.67.12:80;
  server 127.0.0.1:80 backup;
}
server {
     listen       80 ;
     location / {
     proxy_pass http: //websrvs ;
     }

 

BACKUP:

1
2
3
4
5
6
7
8
9
10
upstream websrvs {
  server 172.18.67.11:80;
  server 172.18.67.12:80;
  server 127.0.0.1:80 backup;
}
server {
     listen       80 ;
     location / {
     proxy_pass http: //websrvs ;
     }

 

3.keepalived高可用配置
MASTER:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
! Configuration File  for  keepalived
global_defs {
     notification_email {
  root@localhost
     }
     notification_email_from keepalived@localhost
     smtp_server 127.0.0.1
     smtp_connect_timeout 30
     router_id node1
     vrrp_mcast_group4 224.0.67.67
}
vrrp_script chk_down {
     script  "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"
     interval 1
     weight -5
}
vrrp_script chk_nginx {
     script  "killall -0 nginx && exit 0 || exit 1"
     interval 1
     weight -5
     fall 2
     rise 1
}
vrrp_instance myr {
     state MASTER
     interface eno16777736
     virtual_router_id 167
     priority 100
     advert_int 1
     authentication {
  auth_type PASS
  auth_pass 571f97b2
     }
     virtual_ipaddress {
  172.18.67.33 /16  dev eno16777736
     }
     track_script {
  chk_down
  chk_nginx
     }
     notify_master  "/etc/keepalived/notify.sh master"
     notify_backup  "/etc/keepalived/notify.sh backup"
     notify_fault  "/etc/keepalived/notify.sh fault"
}

 

BACKUP:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
! Configuration File  for  keepalived
global_defs {
     notification_email {
  root@localhost
     }
     notification_email_from keepalived@localhost
     smtp_server 127.0.0.1
     smtp_connect_timeout 30
     router_id node1
     vrrp_mcast_group4 224.0.67.67
}
vrrp_script chk_down {
     script  "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"
     interval 1
     weight -5
}
vrrp_script chk_nginx {
     script  "killall -0 nginx && exit 0 || exit 1"
     interval 1
     weight -5
     fall 2
     rise 1
}
vrrp_instance myr {
     state BACKUP
     interface eno16777736
     virtual_router_id 167
     priority 95
     advert_int 1
     authentication {
  auth_type PASS
  auth_pass 571f97b2
     }
     virtual_ipaddress {
  172.18.67.33 /16  dev eno16777736
     }
     track_script {
  chk_down
  chk_nginx
     }
     notify_master  "/etc/keepalived/notify.sh master"
     notify_backup  "/etc/keepalived/notify.sh backup"
     notify_fault  "/etc/keepalived/notify.sh fault"
}

 

4.通知脚本示例

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
[root@inode2 nginx] # vim notify.sh
#!/bin/bash
#
contact= 'root@localhost'
notify() {
  mailsubject= "$(hostname) to be $1, vip floating"
  mailbody= "$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"
  echo  "$mailbody"  | mail -s  "$mailsubject"  $contact
}
case  $1  in
master)
  notify master
  ;;
backup)
  notify backup
  ;;
fault)
  notify fault
  ;;
*)
  echo  "Usage: $(basename $0) {master|backup|fault}"
  exit  1
  ;;
esac


节点二同样配置


四 启动服务并测试

1.启动后端web服务器

1
2
[root@inode4 ~] # systemctl start httpd
[root@inode5 ~] # systemctl start httpd


为了测试显示效果明显一点,自定义一个访问页面

1
2
[root@inode4 ~] # echo "RS1:172.18.67.11" > /var/www/html/index.html
[root@inode5 ~] # echo "RS2:172.18.67.12" > /var/www/html/index.html

 

2.测试
MASTER:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
[root@inode2 ~] # systemctl start  keepalived
[root@inode2 ~] # systemctl status -l  keepalived
● keepalived.service - LVS and VRRP High Availability Monitor
    Loaded: loaded ( /usr/lib/systemd/system/keepalived .service; disabled; vendor preset: disabled)
    Active: active (running) since Mon 2017-05-15 15:45:20 CST; 3s ago
   Process: 20971 ExecStart= /usr/sbin/keepalived  $KEEPALIVED_OPTIONS (code=exited, status=0 /SUCCESS )
  Main PID: 20972 (keepalived)
    CGroup:  /system .slice /keepalived .service
            ├─20972  /usr/sbin/keepalived  -D
            ├─20973  /usr/sbin/keepalived  -D
            └─20974  /usr/sbin/keepalived  -D
May 15 15:45:20 inode2 Keepalived_healthcheckers[20973]: Opening  file  '/etc/keepalived/keepalived.conf' .
May 15 15:45:20 inode2 Keepalived_healthcheckers[20973]: Configuration is using : 7521 Bytes
May 15 15:45:20 inode2 Keepalived_healthcheckers[20973]: Using LinkWatch kernel netlink reflector...
May 15 15:45:20 inode2 Keepalived_vrrp[20974]: VRRP_Script(chk_nginx) succeeded
May 15 15:45:21 inode2 Keepalived_vrrp[20974]: VRRP_Instance(myr) Transition to MASTER STATE
May 15 15:45:22 inode2 Keepalived_vrrp[20974]: VRRP_Instance(myr) Entering MASTER STATE
May 15 15:45:22 inode2 Keepalived_vrrp[20974]: VRRP_Instance(myr) setting protocol VIPs.
May 15 15:45:22 inode2 Keepalived_vrrp[20974]: VRRP_Instance(myr) Sending gratuitous ARPs on eno16777736  for  172.18.67.33
May 15 15:45:22 inode2 Keepalived_vrrp[20974]: Opening script  file  /etc/keepalived/notify .sh
May 15 15:45:22 inode2 Keepalived_healthcheckers[20973]: Netlink reflector reports IP 172.18.67.33 added
[root@inode2 ~] # ip a l
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
     link /loopback  00:00:00:00:00:00 brd 00:00:00:00:00:00
     inet 127.0.0.1 /8  scope host lo
        valid_lft forever preferred_lft forever
     inet6 ::1 /128  scope host
        valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
     link /ether  00:0c:29:8b:08:6f brd ff:ff:ff:ff:ff:ff
     inet 172.18.67.13 /16  brd 172.18.255.255 scope global eno16777736
        valid_lft forever preferred_lft forever
     inet 172.18.67.33 /16  scope global secondary eno16777736
        valid_lft forever preferred_lft forever
     inet6 fe80::20c:29ff:fe8b:86f /64  scope link tentative dadfailed
        valid_lft forever preferred_lft forever


主节点启动,Entering MASTER STATE,此时我们在客户端进行测试访问

1
2
3
4
5
[root@inode1 ~] # for i in {1..4};do curl http://172.18.67.33;done
RS1:172.18.67.11
RS2:172.18.67.12
RS1:172.18.67.11
RS2:172.18.67.12

 

访问正常,接下来我们启动备用节点的服务器

BACKUP:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
[root@inode3 keepalived] # systemctl start keepalived
[root@inode3 keepalived] # systemctl status -l keepalived
● keepalived.service - LVS and VRRP High Availability Monitor
    Loaded: loaded ( /usr/lib/systemd/system/keepalived .service; disabled; vendor preset: disabled)
    Active: active (running) since Mon 2017-05-15 15:46:51 CST; 3s ago
   Process: 24329 ExecStart= /usr/sbin/keepalived  $KEEPALIVED_OPTIONS (code=exited, status=0 /SUCCESS )
  Main PID: 24330 (keepalived)
    CGroup:  /system .slice /keepalived .service
            ├─24330  /usr/sbin/keepalived  -D
            ├─24331  /usr/sbin/keepalived  -D
            └─24332  /usr/sbin/keepalived  -D
May 15 15:46:51 inode3 Keepalived_vrrp[24332]: Registering Kernel netlink  command  channel
May 15 15:46:51 inode3 Keepalived_vrrp[24332]: Registering gratuitous ARP shared channel
May 15 15:46:51 inode3 Keepalived_vrrp[24332]: Opening  file  '/etc/keepalived/keepalived.conf' .
May 15 15:46:51 inode3 Keepalived_vrrp[24332]: Configuration is using : 66427 Bytes
May 15 15:46:51 inode3 Keepalived_vrrp[24332]: Using LinkWatch kernel netlink reflector...
May 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP_Instance(myr) Entering BACKUP STATE
May 15 15:46:51 inode3 Keepalived_vrrp[24332]: Opening script  file  /etc/keepalived/notify .sh
May 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
May 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP_Script(chk_down) succeeded
May 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP_Script(chk_nginx) succeeded
[root@inode3 keepalived] # ip a l
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
     link /loopback  00:00:00:00:00:00 brd 00:00:00:00:00:00
     inet 127.0.0.1 /8  scope host lo
        valid_lft forever preferred_lft forever
     inet6 ::1 /128  scope host
        valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
     link /ether  00:0c:29:78:24:c3 brd ff:ff:ff:ff:ff:ff
     inet 172.18.67.14 /16  brd 172.18.255.255 scope global eno16777736
        valid_lft forever preferred_lft forever
     inet6 fe80::20c:29ff:fe78:24c3 /64  scope link tentative dadfailed
        valid_lft forever preferred_lft forever

 

此时,我们可以看到备用节点服务器启动后进入了BACKUP状态,Entering BACKUP STATE。接下来我们测试主节点宕机的情形下,我们的服务是否还可用

1
[root@inode2 ~] # systemctl stop keepalived


主节点宕机后我们查看备用节点的状态

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
[root@inode3 keepalived] # systemctl status -l keepalived
● keepalived.service - LVS and VRRP High Availability Monitor
    Loaded: loaded ( /usr/lib/systemd/system/keepalived .service; disabled; vendor preset: disabled)
    Active: active (running) since Mon 2017-05-15 15:46:51 CST; 2min 19s ago
   Process: 24329 ExecStart= /usr/sbin/keepalived  $KEEPALIVED_OPTIONS (code=exited, status=0 /SUCCESS )
  Main PID: 24330 (keepalived)
    CGroup:  /system .slice /keepalived .service
            ├─24330  /usr/sbin/keepalived  -D
            ├─24331  /usr/sbin/keepalived  -D
            └─24332  /usr/sbin/keepalived  -D
May 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
May 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP_Script(chk_down) succeeded
May 15 15:46:51 inode3 Keepalived_vrrp[24332]: VRRP_Script(chk_nginx) succeeded
May 15 15:48:35 inode3 Keepalived_vrrp[24332]: VRRP_Instance(myr) Transition to MASTER STATE
May 15 15:48:36 inode3 Keepalived_vrrp[24332]: VRRP_Instance(myr) Entering MASTER STATE
May 15 15:48:36 inode3 Keepalived_vrrp[24332]: VRRP_Instance(myr) setting protocol VIPs.
May 15 15:48:36 inode3 Keepalived_vrrp[24332]: VRRP_Instance(myr) Sending gratuitous ARPs on eno16777736  for  172.18.67.33
May 15 15:48:36 inode3 Keepalived_vrrp[24332]: Opening script  file  /etc/keepalived/notify .sh
May 15 15:48:36 inode3 Keepalived_healthcheckers[24331]: Netlink reflector reports IP 172.18.67.33 added
May 15 15:48:41 inode3 Keepalived_vrrp[24332]: VRRP_Instance(myr) Sending gratuitous ARPs on eno16777736  for  172.18.67.33
[root@inode3 keepalived] # ip a l
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
     link /loopback  00:00:00:00:00:00 brd 00:00:00:00:00:00
     inet 127.0.0.1 /8  scope host lo
        valid_lft forever preferred_lft forever
     inet6 ::1 /128  scope host
        valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
     link /ether  00:0c:29:78:24:c3 brd ff:ff:ff:ff:ff:ff
     inet 172.18.67.14 /16  brd 172.18.255.255 scope global eno16777736
        valid_lft forever preferred_lft forever
     inet 172.18.67.33 /16  scope global secondary eno16777736
        valid_lft forever preferred_lft forever
     inet6 fe80::20c:29ff:fe78:24c3 /64  scope link tentative dadfailed
        valid_lft forever preferred_lft forever

 

我们发现备用节点由备用状态进入了主状态,并且IP地址也成功绑定至备用节点下。再次进行测试访问

1
2
3
4
5
[root@inode1 ~] # for i in {1..4};do curl http://172.18.67.33;done
RS1:172.18.67.11
RS2:172.18.67.12
RS1:172.18.67.11
RS2:172.18.67.12

测试一台web服务器宕机

1
2
3
4
5
6
[root@inode4 ~] # systemctl stop httpd
[root@inode1 ~] # for i in {1..4};do curl http://172.18.67.33;done
RS2:172.18.67.12
RS2:172.18.67.12
RS2:172.18.67.12
RS2:172.18.67.12


在实际生产环境中后端两台web服务器的内容应该一样的,在这里我们可认为客户端已成功访问到服务器,因此我们可认为这样的架构体现了高可用负载均衡。

 

转载于:https://www.cnblogs.com/Van214/p/8611420.html

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值