- 配置
LVS
服务,至少需要四个节点
server20.com 172.25.23.20
server21.com 172.25.23.21
server22.com 172.25.23.22
server24.com 172.25.23.24
需要关闭各个节点上面的iptables selinux
,并且禁止iptables
开机启动;
- 首先在
server22.com
以及server24.com
上面安装httpd
服务并且启动,
[root@server22 ~]# yum install httpd -y
[root@server22 ~]# /etc/init.d/httpd start
[root@server24 ~]# yum install httpd -y
[root@server24 ~]# /etc/init.d/httpd start
- 确保浏览器访问可以得到下面的访问结果
- 接下来配置
directory
节点 - 首先需要给
directory
上面配置一个VIP
地址
[root@server21 ~]# ip addr add 172.25.23.100/24 dev eth0
[root@server21 ~]# ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:62:3a:95 brd ff:ff:ff:ff:ff:ff
inet 172.25.23.21/24 brd 172.25.23.255 scope global eth0
inet 172.25.23.100/24 scope global secondary eth0
inet6 fe80::5054:ff:fe62:3a95/64 scope link
valid_lft forever preferred_lft forever
- 在提供
IP
地址之后,httpd
服务并不能够正常访问,因为并没有指定合适的httpd
服务规则和后端的httpd
服务节点进行绑定;
[root@server21 ~]# ipvsadm -A -t 172.25.23.100:80 -s rr
[root@server21 ~]# ipvsadm -a -t 172.25.23.100:80 -r 172.25.23.22:80 -g
[root@server21 ~]# ipvsadm -a -t 172.25.23.100:80 -r 172.25.23.24:80 -g
- 可以查看新添加的规则
[root@server21 ~]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.25.23.100:http rr
-> 172.25.23.22:http Route 1 0 0
-> 172.25.23.24:http Route 1 0 0
- 接下来查看是否正确进行调度,以及调度到那个节点上面
[root@server21 ~]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.25.23.100:http rr
-> 172.25.23.22:http Route 1 0 2 [表示两个链接]
-> 172.25.23.24:http Route 1 0 2
- 虽然进行了上述配置,但是服务仍然是不能够正常访问的,接下来在
realserver
上面继续进行配置;
[root@server22 ~]# yum install arptables_jf -y
[root@server22 ~]# arptables -A IN -d 172.25.23.100 -j DROP
[root@server22 ~]# arptables -A OUT -s 172.25.23.100 -j mangle --mangle-ip-s 172.25.23.22
[root@server22 ~]# /etc/init.d/arptables_jf save
Saving current rules to /etc/sysconfig/arptables: [ OK ]
[root@server24 ~]# yum install arptables_jf -y
[root@server24 ~]# arptables -A IN -d 172.25.23.100 -j DROP
[root@server24 ~]# arptables -A OUT -s 172.25.23.100 -j mangle --mangle-ip-s 172.25.23.24
[root@server24 ~]# /etc/init.d/arptables_jf save
Saving current rules to /etc/sysconfig/arptables: [ OK ]
- 接下来需要在两个网卡上面配置别名
[root@server22 ~]# ifconfig eth0:1 172.25.23.100 netmask 255.255.255.0
[root@server24 ~]# ifconfig eth0:1 172.25.23.100 netmask 255.255.255.0
-
然后通过浏览器刷新页面
-
接下来配置
ldirecotry
用于实现对于后端服务的监控状况检查 -
首先需要安装软件包
[root@server21 ~]# yum install ldirectord-3.9.5-3.1.x86_64.rpm
[root@server21 ~]# cp /usr/share/doc/ldirectord-3.9.5/ldirectord.cf /etc/ha.d/
- 修改配置文件
[root@server21 ~]# vim /etc/ha.d/ldirectord.cf
更改或者添加
virtual=172.25.23.100:80
real=172.25.23.22:80 gate
real=172.25.23.24:80 gate
fallback=127.0.0.1:80 gate
service=http
scheduler=rr
#persistent=600
#netmask=255.255.255.255
protocol=tcp
checktype=negotiate
checkport=80
request="index.html"
# receive="Test Page"
# virtualhost=www.x.y.z
- 接下来停止某个节点上面的服务,那么用于的调度请求,就不会被调度到非正常的节点上面
[root@server22 ~]# /etc/init.d/httpd stop
Stopping httpd: [ OK ]
- 服务请求会交给
server24.com
来进行响应
- 为了防止
ldriectory
的单个节点故障,可以使用server20.com
以及server21.com
来配置ldirectory
的高可用集群,这里是结合高可用来进行配置的
[root@server21 ~]# vim /etc/ha.d/haresources
server20.com IPaddr::172.25.23.100/24/eth0 ldirectord httpd
- 同样需要将上面的配置信息复制到另一个节点上面,同时还需要复制
ldirectory
的配置文件
[root@server21 ~]# scp /etc/ha.d/haresources 172.25.23.20:/etc/ha.d/haresources
haresources 100% 5969 5.8KB/s 00:00
[root@server21 ~]# scp /etc/ha.d/ldirectord.cf 172.25.23.20:/etc/ha.d/
ldirectord.cf 100% 8280 8.1KB/s 00:00
- 同时需要确保节点
server20.com
上面是存在ldirectory
程序的
[root@server20 ~]# /etc/init.d/ldirectord
- 因为是配置集群服务,所以资源需要交给集群来进行配置,所以这里需要还原资源的状态
[root@server21 ~]# /etc/init.d/ldirectord stop
Stopping ldirectord... success
[root@server21 ~]# ip addr del 172.25.23.100/24 dev eth0
- 然后在两个节点上面启动
heartbeat
服务
[root@server20 ~]# /etc/init.d/heartbeat start
Starting High-Availability services: INFO: Resource is stopped
Done.
[root@server21 ~]# /etc/init.d/heartbeat start
Starting High-Availability services: INFO: Resource is stopped
Done.
- 接下来写入服务器故障之后,提示用户的信息
[root@server20 ~]# echo <h2>the server is busying please try later</h2> >/var/www/html/index.html
[root@server21 ~]# echo <h2>the server is busying please try later</h2> >/var/www/html/index.html
- 查看
heatbeat
管理的集群资源是否正常运行
[root@server20 ~]# /etc/init.d/ldirectord status
ldirectord for /etc/ha.d/ldirectord.cf is running with pid: 5126
[root@server20 ~]# /etc/init.d/httpd status
httpd (pid 5154) is running...
[root@server20 ~]# ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:03:aa:77 brd ff:ff:ff:ff:ff:ff
inet 172.25.23.20/24 brd 172.25.23.255 scope global eth0
inet 172.25.23.100/24 brd 172.25.23.255 scope global secondary eth0
inet6 fe80::5054:ff:fe03:aa77/64 scope link
valid_lft forever preferred_lft forever
- 查看
ipvs
的信息
[root@server20 ~]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP localhost:http rr
-> 172.25.23.22:http Route 1 0 0
-> 172.25.23.24:http Route 1 0 0
- 接下来关闭节点上面的
httpd
服务
[root@server22 ~]# /etc/init.d/httpd stop
Stopping httpd: [ OK ]
[root@server24 ~]# /etc/init.d/httpd stop
Stopping httpd: [ OK ]
-
接下来通过浏览器查看得到访问失败的页面
-
接下来关闭
heartbeat
并且关闭ldirectory
来配置keepalived
集群
root@server20 ~]# /etc/init.d/heartbeat stop
Stopping High-Availability services: Done.
[root@server20 ~]# chkconfig heartbeat off
[root@server20 ~]# /etc/init.d/ldirectord stop
Stopping ldirectord... success
[root@server20 ~]# chkconfig ldirectord off
root@server20 ~]# /etc/init.d/heartbeat stop
Stopping High-Availability services: Done.
[root@server21 ~]# chkconfig heartbeat off
[root@server21 ~]# /etc/init.d/ldirectord stop
Stopping ldirectord... success
[root@server21 ~]# chkconfig ldirectord off
*keepalived
需要编译进行安装
[root@server20 keepalived-1.2.24]# yum install libnl-devel libnfnetlink-devel-1.0.0-1.el6.x86_64.rpm[这个软件包需要额外下载] -y
[root@server20 keepalived-1.2.24]# ./configure --prefix=/usr/local/keepalived
[root@server20 keepalived-1.2.24]# make
[root@server20 keepalived-1.2.24]# make install
[root@server20 keepalived-1.2.24]# scp -r /usr/local/keepalived/ 172.25.23.21:/usr/local/
[root@server20 keepalived-1.2.24]# ln -sv /usr/local/keepalived/etc/keepalived /etc/
`/etc/keepalived' -> `/usr/local/keepalived/etc/keepalived'
[root@server20 keepalived-1.2.24]# ln -sv /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
`/etc/init.d/keepalived' -> `/usr/local/keepalived/etc/rc.d/init.d/keepalived'
[root@server20 keepalived-1.2.24]# chmod +x /etc/init.d/keepalived
[root@server20 keepalived-1.2.24]# ln -sv /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/keepalived
`/etc/sysconfig/keepalived' -> `/usr/local/keepalived/etc/sysconfig/keepalived'
[root@server20 keepalived-1.2.24]# ln -sv /usr/local/keepalived/sbin/keepalived /sbin/
`/sbin/keepalived' -> `/usr/local/keepalived/sbin/keepalived'
[root@server20 keepalived-1.2.24]# ln -sv /usr/local/keepalived/etc/ /etc/
`/etc/etc' -> `/usr/local/keepalived/etc/'
- 接下来还需要在
server21.com
上面需要进行配置
[root@server21 ~]# ln -sv /usr/local/keepalived/etc/keepalived /etc/
`/etc/keepalived' -> `/usr/local/keepalived/etc/keepalived'
[root@server21 ~]# ln -sv /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
[root@server21 ~]# chmod +x /etc/init.d/keepalived
[root@server21 ~]# ln -sv /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
[root@server21 ~]# ln -sv /usr/local/keepalived/sbin/keepalived /sbin/
- 在两个节点上面尝试启动服务,并且确定是正常的
[root@server21 ~]# /etc/init.d/keepalived start
Starting keepalived: [ OK ]
[root@server21 ~]# /etc/init.d/keepalived stop
Stopping keepalived: [ OK ]
[root@server20 keepalived-1.2.24]# /etc/init.d/keepalived start
Starting keepalived: [ OK ]
[root@server20 keepalived-1.2.24]# /etc/init.d/keepalived stop
Stopping keepalived: [ OK ]
- 接下来需要修改配置文件
[root@server20 ~]# vim /etc/keepalived/keepalived.conf
添加或者修改下面的信息:
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@server20.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.23.100
}
}
virtual_server 172.25.23.100 80 {
delay_loop 6
lb_algo rr
lb_kind DR
# persistence_timeout 50
protocol TCP
real_server 172.25.23.22 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 172.25.23.24 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
- 将这个配置文件复制给
server21.com
[root@server20 ~]# scp /etc/keepalived/keepalived.conf 172.25.23.21:/etc/keepalived/keepalived.conf
keepalived.conf 100% 1037 1.0KB/s 00:00
- 并且进行下面的修改
更改如下:
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@server21.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 50
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.23.100
}
}
virtual_server 172.25.23.100 80 {
delay_loop 6
lb_algo rr
lb_kind DR
# persistence_timeout 50
protocol TCP
real_server 172.25.23.22 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 172.25.23.24 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
- 配置
real server
的httpd
服务正常
[root@server22 ~]# /etc/init.d/httpd start
Starting httpd: httpd: apr_sockaddr_info_get() failed for server22.com
httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
[ OK ]
[root@server24 ~]# /etc/init.d/httpd start
Starting httpd: [ OK ]
keepalived
节点配置启动keepalived
服务
[root@server20 ~]# /etc/init.d/keepalived restart
Stopping keepalived: [ OK ]
Starting keepalived: [ OK ]
[root@server21 ~]# /etc/init.d/keepalived restart
Stopping keepalived: [ OK ]
Starting keepalived: [ OK ]
- 查看日志记录的服务启动信息
Apr 4 22:48:13 server20 Keepalived_vrrp[16094]: Sending gratuitous ARP on eth0 for 172.25.23.100
Apr 4 22:48:13 server20 Keepalived_vrrp[16094]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on eth0 for 172.25.23.100
Apr 4 22:48:13 server20 Keepalived_vrrp[16094]: Sending gratuitous ARP on eth0 for 172.25.23.100
Apr 4 22:48:13 server20 Keepalived_vrrp[16094]: Sending gratuitous ARP on eth0 for 172.25.23.100
Apr 4 22:48:13 server20 Keepalived_vrrp[16094]: Sending gratuitous ARP on eth0 for 172.25.23.100
Apr 4 22:48:13 server20 Keepalived_vrrp[16094]: Sending gratuitous ARP on eth0 for 172.25.23.100
Apr 4 22:48:13 server20 Keepalived_healthcheckers[16093]: Netlink reflector reports IP 172.25.23.100 added
Apr 4 22:48:18 server20 Keepalived_vrrp[16094]: Sending gratuitous ARP on eth0 for 172.25.23.100
Apr 4 22:48:18 server20 Keepalived_vrrp[16094]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on eth0 for 172.25.23.100
Apr 4 22:48:18 server20 Keepalived_vrrp[16094]: Sending gratuitous ARP on eth0 for 172.25.23.100
Apr 4 22:48:18 server20 Keepalived_vrrp[16094]: Sending gratuitous ARP on eth0 for 172.25.23.100
Apr 4 22:48:18 server20 Keepalived_vrrp[16094]: Sending gratuitous ARP on eth0 for 172.25.23.100
Apr 4 22:48:18 server20 Keepalived_vrrp[16094]: Sending gratuitous ARP on eth0 for 172.25.23.100
- 通过浏览器可以正常访问虚拟
IP
提供的服务
- 通过关闭
server20.com
上面的keepalived
来观察服务的迁移过程 server20.com
上面显示服务已经停止
Apr 4 22:57:38 server20 Keepalived_vrrp[16094]: VRRP_Instance(VI_1) removing protocol VIPs.
Apr 4 22:57:38 server20 Keepalived_healthcheckers[16093]: Netlink reflector reports IP 172.25.23.100 removed
Apr 4 22:57:38 server20 Keepalived_healthcheckers[16093]: Removing service [172.25.23.22]:80 from VS [172.25.23.100]:80
Apr 4 22:57:38 server20 Keepalived_healthcheckers[16093]: Removing service [172.25.23.24]:80 from VS [172.25.23.100]:80
Apr 4 22:57:38 server20 Keepalived_healthcheckers[16093]: Stopped
server21.com
上面显示服务正在配置
Apr 4 22:57:45 server21 Keepalived_vrrp[6283]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on eth0 for 172.25.23.100
Apr 4 22:57:45 server21 Keepalived_vrrp[6283]: Sending gratuitous ARP on eth0 for 172.25.23.100
Apr 4 22:57:45 server21 Keepalived_vrrp[6283]: Sending gratuitous ARP on eth0 for 172.25.23.100
Apr 4 22:57:45 server21 Keepalived_vrrp[6283]: Sending gratuitous ARP on eth0 for 172.25.23.100
Apr 4 22:57:45 server21 Keepalived_vrrp[6283]: Sending gratuitous ARP on eth0 for 172.25.23.100
- 地址已经正常配置
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:62:3a:95 brd ff:ff:ff:ff:ff:ff
inet 172.25.23.21/24 brd 172.25.23.255 scope global eth0
inet 172.25.23.100/32 scope global eth0
ipvsadm
的规则也已经转移过来
[root@server21 ~]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.25.23.100:http rr
-> 172.25.23.22:http Route 1 0 0
-> 172.25.23.24:http Route 1 0 0
- 服务的访问同样是正常的;