本文介绍如何通过Nginx和Keepalived搭建高可用集群，包括安装配置过程及测试步骤，确保服务的连续性和稳定性。
说明：
两台服务器ip:192.168.163.128(主) 192.168.163.129(备)
两台服务器分别安装nginx和keepalived
安装配置：
首先安装nginx（yum和编译安装都可以，128和129的安装方式一样，这里以128为例）：
如果没有服务器没有nginx的rpm包则需要先安装扩展源：
yum install -y epel-releaseyum安装nginx ：
yum install -y nginx安装完成后查看nginx的rpm包：
[root@localhost ~]# rpm -qa nginx nginx-1.10.2-1.el6.x86_64
查看nginx安装目录：[root@localhost ~]# rpm -ql nginx /etc/logrotate.d/nginx /etc/nginx/conf.d/default.conf /etc/nginx/conf.d/ssl.conf /etc/nginx/conf.d/virtual.conf /etc/nginx/fastcgi.conf /etc/nginx/fastcgi.conf.default /etc/nginx/fastcgi_params /etc/nginx/fastcgi_params.default /etc/nginx/koi-utf /etc/nginx/koi-win /etc/nginx/mime.types /etc/nginx/mime.types.default /etc/nginx/nginx.conf /etc/nginx/nginx.conf.default /etc/nginx/scgi_params /etc/nginx/scgi_params.default /etc/nginx/uwsgi_params /etc/nginx/uwsgi_params.default /etc/nginx/win-utf /etc/rc.d/init.d/nginx /etc/sysconfig/nginx /usr/lib64/nginx/modules /usr/sbin/nginx /usr/share/doc/nginx-1.10.2 /usr/share/doc/nginx-1.10.2/CHANGES /usr/share/doc/nginx-1.10.2/LICENSE /usr/share/doc/nginx-1.10.2/README /usr/share/doc/nginx-1.10.2/README.dynamic /usr/share/doc/nginx-1.10.2/UPGRADE-NOTES-1.0-to-1.10 /usr/share/man/man3/nginx.3pm.gz /usr/share/man/man8/nginx.8.gz /usr/share/nginx/html/404.html /usr/share/nginx/html/50x.html /usr/share/nginx/html/index.html /usr/share/nginx/html/nginx-logo.png /usr/share/nginx/html/poweredby.png /usr/share/vim/vimfiles/ftdetect/nginx.vim /usr/share/vim/vimfiles/indent/nginx.vim /usr/share/vim/vimfiles/syntax/nginx.vim /var/lib/nginx /var/lib/nginx/tmp /var/log/nginx
主要关注nginx配置和html文件（标红部分）
nginx配置文件：
[root@localhost ~]# vim /etc/nginx/nginx.conf user root; worker_processes 1; #error_log logs/error.log; #error_log logs/error.log notice; #error_log logs/error.log info; #pid logs/nginx.pid; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; #log_format main '$remote_addr - $remote_user [$time_local] "$request" ' # '$status $body_bytes_sent "$http_referer" ' # '"$http_user_agent" "$http_x_forwarded_for"'; #access_log logs/access.log main; sendfile on; #tcp_nopush on; #keepalive_timeout 0; keepalive_timeout 65; #gzip on; server { listen 80; server_name localhost; #charset koi8-r; #access_log logs/host.access.log main; location / { root /usr/share/nginx/html; index index.html index.htm; } #error_page 404 /404.html; # redirect server error pages to the static page /50x.html error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } }
修改默认访问页内容：
<!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>Welcome to nginx!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> 192.168.163.128 的标题 <!-- 这里添加ip地址 以便于后面测试更直观的查看切换结果--> <h1>Welcome to nginx! 128</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html>
启动nginx：
查看nginx进程：service nginx start[root@localhost ~]# ps -ef|grep nginx root 1811 1 0 19:41 ? 00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf root 1813 1811 0 19:41 ? 00:00:00 nginx: worker process root 10026 1512 0 20:50 pts/0 00:00:00 vim /usr/share/nginx/html/index.html root 10628 1529 0 20:55 pts/1 00:00:00 grep nginx
yum安装keepalived服务(主备安装方式相同)：
yum install -y keepalived查看安装路径：
[root@localhost ~]# rpm -ql keepalived /etc/keepalived /etc/keepalived/keepalived.conf /etc/rc.d/init.d/keepalived /etc/sysconfig/keepalived /usr/bin/genhash /usr/libexec/keepalived /usr/sbin/keepalived /usr/share/doc/keepalived-1.2.13 /usr/share/doc/keepalived-1.2.13/AUTHOR /usr/share/doc/keepalived-1.2.13/CONTRIBUTORS /usr/share/doc/keepalived-1.2.13/COPYING /usr/share/doc/keepalived-1.2.13/ChangeLog /usr/share/doc/keepalived-1.2.13/NOTE_vrrp_vmac.txt /usr/share/doc/keepalived-1.2.13/README /usr/share/doc/keepalived-1.2.13/TODO /usr/share/doc/keepalived-1.2.13/VERSION /usr/share/doc/keepalived-1.2.13/keepalived.conf.SYNOPSIS /usr/share/doc/keepalived-1.2.13/samples /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.HTTP_GET.port /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.IPv6 /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.SMTP_CHECK /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.SSL_GET /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.fwmark /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.inhibit /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.misc_check /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.misc_check_arg /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.quorum /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.sample /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.status_code /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.track_interface /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.virtual_server_group /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.virtualhost /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.vrrp /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.vrrp.localcheck /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.vrrp.lvs_syncd /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.vrrp.routes /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.vrrp.scripts /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.vrrp.static_ipaddress /usr/share/doc/keepalived-1.2.13/samples/keepalived.conf.vrrp.sync /usr/share/doc/keepalived-1.2.13/samples/sample.misccheck.smbcheck.sh /usr/share/man/man1/genhash.1.gz /usr/share/man/man5/keepalived.conf.5.gz /usr/share/man/man8/keepalived.8.gz /usr/share/snmp/mibs/KEEPALIVED-MIB.txt
配置文件路径：/etc/keepalived/keepalived.conf
清空配置文件：
> /etc/keepalived/keepalived.conf主服务器的配置文件：
global_defs { ###全局定义参数 notification_email { ###出现问题时给邮箱发邮件 zhang@eflying.com ###由于本实验用不到发邮件的功能 所以可以不用定义 或者随便写一下 } notification_email_from root@eflying.com ###由哪一个邮箱发出去 可以引用第三方 smtp_server 127.0.0.1 ###sendmail 服务 smtp_connect_timeout 30 ###超时时间 s router_id zhang129 ##### 标识本节点的字条串，通常为 hostname } vrrp_script chk_nginx { ### 执行脚本 检测nginx服务是否正常 script "/usr/local/check_ng.sh" ###定义检查脚本路径并编写检查脚本来检查这个服务是否正常 如果不正常 要把它启动起来 名字可以自己定义 脚本名字和这里定义的保持一直就行 interval 3 ###检测的间断是3秒钟 weight -20 ###权重 } vrrp_instance VI_1 { state MASTER ## 主节点为 MASTER， 对应的备份节点为 BACKUP interface eth0 ## 绑定虚拟 IP 的网络接口，与本机 IP 地址所在的网络接口相同， eth0 virtual_router_id 51 ## 虚拟路由的 ID 号， 两个节点设置必须一样， 可选 IP 最后一段使用, 相同的 VRID 为一个组，这将决定多播的 MAC 地址 mcast_src_ip 192.168.163.129 ## 本机 IP 地址 nopreempt ## 优先级高的设置 nopreempt 解决异常恢复后再次抢占的问题 priority 100 ## 节点优先级， 值范围 0-254， MASTER 要比 BACKUP 高 advert_int 1 ## 组播信息发送间隔，两个节点设置必须一样， 默认 1s authentication { ##设置验证信息 两个节点必须设置一致 auth_type PASS ##密码类型 auth_pass zhang ## 密码 ，真实生产，按需求对应该过来 } virtual_ipaddress { 192.168.163.130 # 虚拟 IP 池, 两个节点设置必须一样 即vip 可以定义多个 } track_script { ## 将 track_script 块加入 instance 配置块 chk_nginx ## 执行 Nginx 监控的服务 } }
check_ng.sh脚本：
[root@localhost ~]# vim /usr/local/check_ng.sh #!/bin/bash d=`date --date today +%Y%m%d_%H:%M:%S` #计算nginx进程数量 n=`ps -C nginx --no-heading|wc -l` #如果进程为0，则启动nginx，并且再次检测nginx进程数量， #如果还为0，说明nginx无法启动，此时需要关闭keepalived if [ $n -eq 0 ]; then service nginx start n2=`ps -C nginx --no-heading|wc -l` if [ $n2 -eq 0 ]; then echo "$d nginx down,keepalived will stop" >> /var/log/check_ng.log service keepalived stop fi fi
此处注意脚本中的` ,不是英文'命令替换.`command` 结构使字符(`) [译者注：这个字符不是单引号，而是在标准美国键盘上的ESC键下面，在字符1左边，在TAB键上面的那个键，要特别留心] 引住的命令（command）执行结果能赋值给一个变量。它也被称为后引号（backquotes）或是斜引号（backticks）添加脚本执行权限：chmod 755 /usr/local/check_ng.sh
到此 ，主服务器的nginx和keepalived已经安装完毕；备服务器的nginx和keepalived安装同主服务器安装方法注意修改nginx的默认访问页面/usr/share/nginx/html/index.html 内容： 添加 备用服务器的ip，以便查看测试结果备用keepalived的配置文件/etc/keepalived/keepalived.conf：[root@localhost ~]# vim /etc/keepalived/keepalived.conf global_defs { notification_email { zhang@eflying.com } notification_email_from root@eflying.com smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id zhang129 } vrrp_script chk_nginx { script "/usr/local/check_ng.sh" interval 3 weight -20 } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 51 mcast_src_ip 192.168.163.129 nopreempt priority 90 advert_int 1 authentication { auth_type PASS auth_pass zhang } virtual_ipaddress { 192.168.163.130 } track_script { chk_nginx } }
监控脚本/usr/local/check_ng.sh同主服务器一致；
此时，主备服务器的nginx和keepalived都已安装完成
------------------------------------------------------------------------------
测试：
首先开启主服务器的nginx：
service nginx start访问主服务器nginx：
开启主服务器的keepalived：
service keepalived start查看keepalived日志：
[root@localhost local]# tail -f /var/log/messages Apr 28 19:43:36 localhost Keepalived_vrrp[7084]: Using LinkWatch kernel netlink reflector... Apr 28 19:43:36 localhost Keepalived_vrrp[7084]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)] Apr 28 19:43:36 localhost Keepalived_vrrp[7084]: VRRP_Script(chk_nginx) succeeded ####脚本运行成功 Apr 28 19:43:36 localhost Keepalived_vrrp[7084]: VRRP_Instance(VI_1) Transition to MASTER STATE Apr 28 19:43:36 localhost Keepalived_vrrp[7084]: VRRP_Instance(VI_1) Received lower prio advert, forcing new election Apr 28 19:43:37 localhost Keepalived_vrrp[7084]: VRRP_Instance(VI_1) Entering MASTER STATE Apr 28 19:43:37 localhost Keepalived_vrrp[7084]: VRRP_Instance(VI_1) setting protocol VIPs. Apr 28 19:43:37 localhost Keepalived_vrrp[7084]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.163.130 Apr 28 19:43:37 localhost Keepalived_healthcheckers[7083]: Netlink reflector reports IP 192.168.163.130 added Apr 28 19:43:42 localhost Keepalived_vrrp[7084]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.163.130关闭主服务器的nginx，查看nginx进程发现依然有nginx进程，因为keepalived监控脚本执行重启了nginx[root@localhost ~]# !ps ps -ef|grep nginx root 1811 1 0 19:41 ? 00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf root 1813 1811 0 19:41 ? 00:00:00 nginx: worker process root 10026 1512 0 20:50 pts/0 00:00:00 vim /usr/share/nginx/html/index.html root 19095 1529 0 22:06 pts/1 00:00:00 grep nginx [root@localhost ~]# ps -ef|grep keepalived root 1834 1 0 19:42 ? 00:00:00 /usr/sbin/keepalived -D root 1835 1834 0 19:42 ? 00:00:00 /usr/sbin/keepalived -D root 1837 1834 0 19:42 ? 00:00:04 /usr/sbin/keepalived -D root 19115 1529 0 22:06 pts/1 00:00:00 grep keepalived [root@localhost ~]# service nginx stop 停止 nginx： [确定] [root@localhost ~]# ps -ef|grep keepalived root 1834 1 0 19:42 ? 00:00:00 /usr/sbin/keepalived -D root 1835 1834 0 19:42 ? 00:00:00 /usr/sbin/keepalived -D root 1837 1834 0 19:42 ? 00:00:04 /usr/sbin/keepalived -D root 19194 1529 0 22:06 pts/1 00:00:00 grep keepalived [root@localhost ~]# ps -ef|grep nginx root 10026 1512 0 20:50 pts/0 00:00:00 vim /usr/share/nginx/html/index.html root 19187 1 0 22:06 ? 00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf root 19189 19187 0 22:06 ? 00:00:00 nginx: worker process root 19202 1529 0 22:06 pts/1 00:00:00 grep nginx
此时访问vip地址，发现访问到128(主服务器)的nginx，说明keepalived配置成功；
开启备用服务器nginx：
service nginx start访问备用服务器下的nginx的ip：
开启备用服务器的keepalived：
service keepalived start访问vip：
关闭主服务器的keepalived服务，查看主服务器的keepalived日志：
[root@localhost ~]# service keepalived stop 停止 keepalived： [确定] [root@localhost ~]# [root@localhost local]# tail -f /var/log/messages Apr 28 22:22:25 localhost Keepalived[7081]: Stopping Keepalived v1.2.13 (03/19,2015) Apr 28 22:22:25 localhost Keepalived_vrrp[7084]: VRRP_Instance(VI_1) sending 0 priority Apr 28 22:22:25 localhost Keepalived_vrrp[7084]: VRRP_Instance(VI_1) removing protocol VIPs. Apr 28 22:22:25 localhost Keepalived_healthcheckers[7083]: Netlink reflector reports IP 192.168.163.130 removed
可以看出 vip在主服务器的keepalived绑定移除了；
此时查看备用服务器的keepalived日志:[root@localhost ~]# !tail tail -f /var/log/messages Apr 28 22:22:21 localhost Keepalived_vrrp[1837]: VRRP_Instance(VI_1) Transition to MASTER STATE Apr 28 22:22:22 localhost Keepalived_vrrp[1837]: VRRP_Instance(VI_1) Entering MASTER STATE Apr 28 22:22:22 localhost Keepalived_vrrp[1837]: VRRP_Instance(VI_1) setting protocol VIPs. Apr 28 22:22:22 localhost Keepalived_healthcheckers[1835]: Netlink reflector reports IP 192.168.163.130 added Apr 28 22:22:22 localhost Keepalived_vrrp[1837]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.163.130 Apr 28 22:22:27 localhost Keepalived_vrrp[1837]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.163.130发现备用服务器的keepalived绑定了vip
访问vip：
访问到了129的nginx，说明主备切换成功；
此时关闭主服务器的nginx，开启主服务器的keepalived：
service keepalived start查看nginx进程，发现nginx有启动了；
查看主服务器keepalived日志，发现主服务器keepalived又绑定了vip：
[root@localhost local]# tail -f /var/log/messages Apr 28 22:22:25 localhost Keepalived_healthcheckers[7083]: Netlink reflector reports IP 192.168.163.130 removed Apr 28 22:29:40 localhost Keepalived[26222]: Starting Keepalived v1.2.13 (03/19,2015) Apr 28 22:29:40 localhost Keepalived[26223]: Starting Healthcheck child process, pid=26225 Apr 28 22:29:40 localhost Keepalived[26223]: Starting VRRP child process, pid=26226 Apr 28 22:29:40 localhost Keepalived_vrrp[26226]: Netlink reflector reports IP 192.168.163.128 added Apr 28 22:29:40 localhost Keepalived_healthcheckers[26225]: Netlink reflector reports IP 192.168.163.128 added Apr 28 22:29:40 localhost Keepalived_vrrp[26226]: Netlink reflector reports IP 172.17.42.1 added Apr 28 22:29:40 localhost Keepalived_vrrp[26226]: Netlink reflector reports IP fe80::20c:29ff:fef6:e5f0 added Apr 28 22:29:40 localhost Keepalived_vrrp[26226]: Netlink reflector reports IP fe80::2ccf:b6ff:fe1c:2abd added Apr 28 22:29:40 localhost Keepalived_vrrp[26226]: Registering Kernel netlink reflector Apr 28 22:29:40 localhost Keepalived_vrrp[26226]: Registering Kernel netlink command channel Apr 28 22:29:40 localhost Keepalived_vrrp[26226]: Registering gratuitous ARP shared channel Apr 28 22:29:40 localhost Keepalived_vrrp[26226]: Opening file '/etc/keepalived/keepalived.conf'. Apr 28 22:29:40 localhost Keepalived_healthcheckers[26225]: Netlink reflector reports IP 172.17.42.1 added Apr 28 22:29:40 localhost Keepalived_healthcheckers[26225]: Netlink reflector reports IP fe80::20c:29ff:fef6:e5f0 added Apr 28 22:29:40 localhost Keepalived_healthcheckers[26225]: Netlink reflector reports IP fe80::2ccf:b6ff:fe1c:2abd added Apr 28 22:29:40 localhost Keepalived_healthcheckers[26225]: Registering Kernel netlink reflector Apr 28 22:29:40 localhost Keepalived_vrrp[26226]: Truncating auth_pass to 8 characters Apr 28 22:29:40 localhost Keepalived_healthcheckers[26225]: Registering Kernel netlink command channel Apr 28 22:29:40 localhost Keepalived_vrrp[26226]: Configuration is using : 64123 Bytes Apr 28 22:29:40 localhost Keepalived_vrrp[26226]: Using LinkWatch kernel netlink reflector... Apr 28 22:29:40 localhost Keepalived_healthcheckers[26225]: Opening file '/etc/keepalived/keepalived.conf'. Apr 28 22:29:40 localhost Keepalived_healthcheckers[26225]: Configuration is using : 6433 Bytes Apr 28 22:29:40 localhost Keepalived_healthcheckers[26225]: Using LinkWatch kernel netlink reflector... Apr 28 22:29:40 localhost Keepalived_vrrp[26226]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)] Apr 28 22:29:40 localhost Keepalived_vrrp[26226]: VRRP_Script(chk_nginx) succeeded Apr 28 22:29:40 localhost Keepalived_vrrp[26226]: VRRP_Instance(VI_1) Transition to MASTER STATE Apr 28 22:29:40 localhost Keepalived_vrrp[26226]: VRRP_Instance(VI_1) Received lower prio advert, forcing new election Apr 28 22:29:41 localhost Keepalived_vrrp[26226]: VRRP_Instance(VI_1) Entering MASTER STATE Apr 28 22:29:41 localhost Keepalived_vrrp[26226]: VRRP_Instance(VI_1) setting protocol VIPs. Apr 28 22:29:41 localhost Keepalived_vrrp[26226]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.163.130 Apr 28 22:29:41 localhost Keepalived_healthcheckers[26225]: Netlink reflector reports IP 192.168.163.130 added Apr 28 22:29:46 localhost Keepalived_vrrp[26226]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.163.130此时访问vip：
切换成功！！！！！
到此keepalived高可用搭建测试完成！！！
keepalived启动后查看vip绑定：[root@localhost local]# ip add 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:f6:e5:f0 brd ff:ff:ff:ff:ff:ff inet 192.168.163.128/24 brd 192.168.163.255 scope global eth0 inet inet 192.168.163.130/32 scope global eth0 inet6 fe80::20c:29ff:fef6:e5f0/64 scope link valid_lft forever preferred_lft forever 3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN link/ether 2e:cf:b6:1c:2a:bd brd ff:ff:ff:ff:ff:ff inet 172.17.42.1/16 scope global docker0 inet6 fe80::2ccf:b6ff:fe1c:2abd/64 scope link valid_lft forever preferred_lft forever
扫一扫
2667
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
