配置nginx的负载均衡keepalived和nfs磁盘共享

环境说明

3台虚拟机CentOS 6.5 x86_64,安装好hengdalnmp.zip (lnmp1.3)

主nginx负载均衡器:192.168.223.130  (通过keepalived配置了VIP:192.168.223.160供外使用)
副nginx负载均衡器:192.168.223.133(通过keepalived配置了VIP:192.168.223.160供外使用)

后端代理的web服务器:
192.168.223.143:8112       (通过nfs 把项目目录下的 storage共享)
192.168.223.133:18112     (通过nfs 共享192.168.223.143的项目目录下的 storage)

1. 安装部署NFS服务端(192.168.223.143)

Step-1:安装nfs-utils和rpcbind,运行以下命令:
yum install -y nfs-utils rpcbind
 
Step-2:为NFS指定固定端口,运行以下命令:

vi /etc/sysconfig/nfs

搜索和设置如下所示的端口配置:
RQUOTAD_PORT=30001
LOCKD_TCPPORT=30002
LOCKD_UDPPORT=30002
MOUNTD_PORT=30003
STATD_PORT=30004

 
Step-3:开放防火墙中的上述端口,运行以下命令:
iptables  -I INPUT -p tcp --dport 111 -j ACCEPT
iptables  -I INPUT -p udp --dport 111 -j ACCEPT
iptables  -I INPUT -p tcp --dport 2049 -j ACCEPT
iptables  -I INPUT -p udp --dport 2049 -j ACCEPT
iptables  -I INPUT -p tcp --dport 30001:30004 -j ACCEPT
iptables  -I INPUT -p udp --dport 30001:30004 -j ACCEPT
service iptables save
service iptables restart
 

Step-4:设置SELinux为许可状态,运行以下命令:

vi /etc/selinux/config

将上述文件中的
SELINUX=enforcing
替换为
SELINUX=permissive

保存上述文件之后,运行以下命令:
setenforce 0

 
Step-5:创建共享目录,运行以下命令:

mkdir -p /home/www/wwwroot/cjcbwg/storage

上述命令将建立共享目录/home/www/wwwroot/cjcbwg/storage。

 
Step-6:配置exports文件,运行以下命令:

vi /etc/exports

在上述文件的末尾新增一行,如下所示:
/home/www/wwwroot/cjcbwg/storage 192.168.223.133(rw,sync,no_root_squash)
/home/www/wwwroot/cjcbwg/storage *(ro)

这一行表示只有192.168.223.133客户端能够以读写权限挂载共享目录,其他客户端只能以只读权限挂载。

 
Step-7:启动NFS相关服务,运行以下命令:
chkconfig nfs on
chkconfig rpcbind on
service rpcbind start
service nfs start
 
Step-8:检查NFS的相关端口是否已经启用,运行以下命令:
service iptables status
rpcinfo -p localhost

接着部署陈家祠项目 于 /home/www/wwwroot/cjcbwg

2. 安装部署NFS客户端(192.168.223.133)

NFS客户端不需要启动NFS服务,但需要安装nfs-utils,运行以下命令:
yum install -y nfs-utils
Step-1:确定挂载点,运行以下命令:

showmount -e 192.168.223.143

-e选项显示NFS服务端的导出列表。

 
Step-2:创建挂载目录,运行以下命令:

mkdir -p /home/www/wwwroot/cjcbwg/storage

其中,/home/www/wwwroot/cjcbwg/storage为共享目录的挂载点目录。

 
Step-3:挂载共享目录,运行以下命令:

mount -t nfs 192.168.223.143:/home/www/wwwroot/cjcbwg/storage /home/www/wwwroot/cjcbwg/storage

其中,-t选项用于指定文件系统的类型为nfs。

 
Step-4:共享目录使用结束之后,卸载共享目录,运行以下命令:
umount /home/www/wwwroot/cjcbwg/storage
 
开机自动挂载
 
向fstab文件中添加共享目录的挂载条目,即可实现开机自动挂载,但是随后与NFS服务端的连接将始终处于活动状态。运行以下命令:
mkdir -p /home/www/wwwroot/cjcbwg/storage
vi /etc/fstab在上述文件末尾加入共享目录的挂载条目,如下所示:
192.168.223.143:/home/www/wwwroot/cjcbwg/storage /home/www/wwwroot/cjcbwg/storage nfs defaults 0 0
其中,第5个字段设置为0表示共享目录的文件系统不需要使用dump命令进行转储,第6个字段设置为0表示共享目录的文件系统不需要使用fsck命令进行检查。
 接着部署陈家祠项目 于 /home/www/wwwroot/cjcbwg
 
 
 

3. 配置nginx反向代理(192.168.223.130,192.168.223.133)

nginx.conf

在http{}内配置

upstream cjcbwgs {  
		server 192.168.223.143:8112;
		server 192.168.223.133:18112;  
		ip_hash;
	} 

server
    {
        listen 8112 default_server;
        #listen [::]:80 default_server ipv6only=on;
        server_name www.cjcbwg160.com;
        location / {

		proxy_pass   http://cjcbwgs;
		proxy_set_header Host $host:$server_port; 
		proxy_set_header X-Real-IP $remote_addr;
		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        } 

    }

 

 

4. 安装部署keepalived(主192.168.223.130,从192.168.223.133)

-------------------------主从都安装keepalived 设置vrrp------------------------------------------

yum install curl gcc openssl-devel libnl3-devel net-snmp-devel

yum install keepalived

chkconfig keepalived on

开机自动
vim /etc/sysconfig/iptables

添加
-A INPUT -d 192.168.223.160/32 -j ACCEPT
-A INPUT -d 224.0.0.18 -j ACCEPT
-A INPUT -p vrrp -j ACCEPT

重启iptables service iptables restart

-------------------------主从都安装keepalived 设置vrrp------------------------------------------

 

-------------------------主keepalived服务器------------------------------------------
拷贝3个文件(底部附件)
keepalived.conf.master
check_nginx.sh
notify.sh
到/etc/keepalived下

keepalived.conf.master重命名成 keepalived.conf

chmod +x /etc/keepalived/check_nginx.sh
chmod +x /etc/keepalived/notify.sh

如果需要修改3个文件里的IP (演示拿虚拟机Ip演示的)
-------------------------主keepalived服务器------------------------------------------

 

-------------------------从keepalived服务器------------------------------------------
拷贝3个文件(底部附件)
keepalived.conf.backup
check_nginx.sh
notify.sh
到/etc/keepalived下

keepalived.conf.backup重命名成 keepalived.conf

chmod +x /etc/keepalived/check_nginx.sh
chmod +x /etc/keepalived/notify.sh

如果需要修改3个文件里的IP (演示拿虚拟机Ip演示的)
-------------------------从keepalived服务器------------------------------------------

 

 

最后主从都重启,测试是否配置成功

附件点击下载

如果附件地址失效的话,可以去码云git库里找


nginx.conf

user  www www;

worker_processes auto;

error_log  /home/www/wwwlogs/nginx_error.log  info;

pid        /usr/local/nginx/logs/nginx.pid;

#Specifies the value for maximum file descriptors that can be opened by this process.
worker_rlimit_nofile 51200;

events
    {
        use epoll;
        worker_connections 51200;
        multi_accept on;
    }

http
    {
        include       mime.types;
        default_type  application/octet-stream;

        server_names_hash_bucket_size 128;
        client_header_buffer_size 32k;
        large_client_header_buffers 4 32k;
        client_max_body_size 50m;

        sendfile   on;
        tcp_nopush on;

        keepalive_timeout 60;

        tcp_nodelay on;

        fastcgi_connect_timeout 300;
        fastcgi_send_timeout 300;
        fastcgi_read_timeout 300;
        fastcgi_buffer_size 64k;
        fastcgi_buffers 4 64k;
        fastcgi_busy_buffers_size 128k;
        fastcgi_temp_file_write_size 256k;

        gzip on;
        gzip_min_length  1k;
        gzip_buffers     4 16k;
        gzip_http_version 1.1;
        gzip_comp_level 2;
        gzip_types     text/plain application/javascript application/x-javascript text/javascript text/css application/xml application/xml+rss;
        gzip_vary on;
        gzip_proxied   expired no-cache no-store private auth;
        gzip_disable   "MSIE [1-6]\.";

        #limit_conn_zone $binary_remote_addr zone=perip:10m;
        ##If enable limit_conn_zone,add "limit_conn perip 10;" to server section.

        server_tokens off;
        access_log off;
		
		
	upstream webservers {  
		server 192.168.223.143:80;  
		server 192.168.223.133:80;
		#ip_hash;
	}  

	upstream cjcbwgs {  
		server 192.168.223.143:8112;
		server 192.168.223.133:18112;  
		ip_hash;
	} 

server
    {
        listen 80 default_server;
        #listen [::]:80 default_server ipv6only=on;
        server_name www.hengdawb.com;
        location / {  
            proxy_pass   http://webservers;  
        } 

    }

server
    {
        listen 8112 default_server;
        #listen [::]:80 default_server ipv6only=on;
        server_name www.cjcbwg160.com;
        location / {

		proxy_pass   http://cjcbwgs;
		proxy_set_header Host $host:$server_port; 
		proxy_set_header X-Real-IP $remote_addr;
		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        } 

    }

include vhost/*.conf;
}

 

keepalived.conf.master

#ConfigurationFile for keepalived  
global_defs {  
	notification_email {                        ######定义接受邮件的邮箱  
		 86529448@qq.com  
	}  
	notification_email_from lvshuocool@qq.com    ######定义发送邮件的邮箱  
	smtp_server mail.qq.com  
	smtp_connect_timeout 10  
}

  
vrrp_script check_nginx {                   ######定义监控nginx的脚本  
	script "/etc/keepalived/check_nginx.sh"  
	interval 2                            ######监控时间间隔  
	weight 2                              ######负载参数  
}  

vrrp_instance vrrptest {                 ######定义vrrptest实例  
	state MASTER               ######服务器状态  
	interface eth0                     ######使用的接口  
	virtual_router_id 51               ######虚拟路由的标志,一组lvs的虚拟路由标识必须相同,这样才能切换  
	priority 100                      ######服务启动优先级,值越大,优先级越高,BACKUP 不能大于MASTER  
	
	unicast_src_ip  192.168.223.130         ##source ip
	unicast_peer {
		        192.168.223.133               ##dest ip
	}
	
	advert_int 1                        ######服务器之间的存活检查时间  
	authentication {  
		auth_type PASS                     ######认证类型  
		auth_pass lvshuo       ######认证密码,一组lvs 服务器的认证密码必须一致  
	}  
	track_script {                              ######执行监控nginx进程的脚本  
		check_nginx  
	}  
	virtual_ipaddress {                         ######虚拟IP地址  
		192.168.223.160  
	} 
	

	notify_master "/etc/keepalived/notify.sh master"  
	notify_backup "/etc/keepalived/notify.sh backup"  
	notify_fault "/etc/keepalived/notify.sh fault" 
}

 

keepalived.conf.backup

#ConfigurationFile for keepalived  
global_defs {  
	notification_email {                        ######定义接受邮件的邮箱  
		 86529448@qq.com  
	}  
	notification_email_from lvshuocool@qq.com    ######定义发送邮件的邮箱  
	smtp_server mail.qq.com  
	smtp_connect_timeout 10     
}

  
vrrp_script check_nginx {                   ######定义监控nginx的脚本  
	script "/etc/keepalived/check_nginx.sh"  
	interval 2                            ######监控时间间隔  
	weight 2                              ######负载参数  
}  

vrrp_instance vrrptest {                 ######定义vrrptest实例  
	state BACKUP               ######服务器状态  
	interface eth0                     ######使用的接口  
	virtual_router_id 51               ######虚拟路由的标志,一组lvs的虚拟路由标识必须相同,这样才能切换  
	priority 90                      ######服务启动优先级,值越大,优先级越高,BACKUP 不能大于MASTER  
	
	unicast_src_ip  192.168.223.133         ##source ip
	unicast_peer {
		        192.168.223.130               ##dest ip
	}
	
	advert_int 1                        ######服务器之间的存活检查时间  
	authentication {  
		auth_type PASS                     ######认证类型  
		auth_pass lvshuo       ######认证密码,一组lvs 服务器的认证密码必须一致  
	}  
	track_script {                              ######执行监控nginx进程的脚本  
		check_nginx  
	}  
	virtual_ipaddress {                         ######虚拟IP地址  
		192.168.223.160  
	}
	
	
	notify_master "/etc/keepalived/notify.sh master"  
	notify_backup "/etc/keepalived/notify.sh backup"  
	notify_fault "/etc/keepalived/notify.sh fault" 
}

 

check_nginx.sh

if [ "$(ps -ef | grep "nginx: master process"| grep -v grep )" == "" ]  
 then  
    /etc/init.d/keepalived stop  
 else  
    echo "nginx is running"  
fi 

 

notify.sh

#!/bin/bash
# Author: lvshuo <lvshuocool@foxmail.com>
# description: An example of notify script
# 

#get ip
hostip=`/sbin/ifconfig -a|grep inet|grep -v 127.0.0.1|grep -v inet6|awk '{print $2}'|tr -d "addr:"`

email=86529448@qq.com
subject='Keepalived Notification'



notify() {
   curl -s -o /dev/null "http://47.93.81.30:5678/api/users/send_email?p=t&email=$email&subject=$subject&content=$content"
}

case $1 in
    master)
	content="$hostip switch master"
        notify content
        exit 0
    ;;
    backup)
        content="$hostip switch backup"
        notify content
        exit 0
    ;;
    fault)
        content="$hostip has fault"
        notify content
        exit 0
    ;;
    *)
        echo 'Usage: `basename $0` {master|backup|fault}'
        exit 1
    ;;
esac

 

 

 

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值