Haproxy+keepalivd+varnish+LAMP
目 录
实验要求
实验前提
实验环境设计
拓扑结构
ip网络规划
基础环境配置
一、LAMP配置
1、HTTPD配置
2、Discuz安装
二、Haproxy配置:
Haproxy日志输出配置
三、Keepalived配置
四、Varnish 配置
五、修改web服务器日志配置
六、测试
1、客户端访问测试
2、客户端访问haprxoy监控页面测试
3、varnish缓存命中查看
4、后端服务器日志查看
5、重要 假设haproxy 不设置会话保持就会出现下面的情况
七、总结
实验要求
1、 动静分离部署Discuz,动静要实现负载均衡,注意会话问题;
2、 Haproxy和后端主机间添加Vanish缓存;
3、 Hapeoxy设计要求
a、动静分离;
b、stats page ,要求仅能通过本机主机使用管理接口;
c、粉笔考虑不同服务组的调度算法;
d、Haporxy高可用;
4、 给出拓扑,写成博客;
实验前提
实验前需要关闭iptables及selinux;
实验前做好时间同步
实验环境设计
拓扑结构
根据以上要求本次实验给出以下拓扑:
ip网络规划
根据拓扑需求以下为各服务器网络配置:
注:1、因笔者笔记本性能问题dynamic-web组与static-web组分别用一台机器模拟;httpd虚拟主机实现;
2、本实验环境未做mysql的高可用集群,因此本实验环境,将mysql安装在172.16.76.30服务器仅作实验环境需求;
客户端ip:172.16.250.186;
Haproxy1:172.16.76.10;
Haproxy1:172.16.76.20;
KeepalivedVIP1:172.16.76.100;
KeepalivedVIP2:172.16.76.110;
Varnish:172.16.76.60;
dynamic-web1:172.16.76.30:80; dynamic-web1:172.16.76.30:8080;
static-web1:172.16.76.40:80;同上
static-web2:172.16.76.40:8080;
Mysql:172.16.76.30:3306;
注:以下主机名对应实验环境响应服务器
node1:172.16.76.10;
node2:172.16.76.20;
node3:172.16.76.30;
node4:172.16:76.40;
node6:172.16.76.60;
基础环境配置
node1:
[root@node1~]# yum install haproxy keepalived -y
node2:
[root@node2~]# yum install haproxy keepalived -y
node3:
[root@node3~]yum install mariadb-server httpd -y
node4:
[root@node4~]yum install httpd –y
node6:
[root@node6~]# yum install varnish -y
一、LAMP配置
1、HTTPD配置;
首先现在先配置后端web服务;
node3:
[root@node3 ~]# cd /etc/httpd/conf.d/
[root@node3 conf.d]# vim vhost.conf
<VirtualHost 172.16.76.30:8080>
DocumentRoot/apps/www/html/
ServerNamewww.linuxinfo.top
ErrorLoglogs/bbs-8080.error_log
Customloglogs/bbs-8080.access_log combined
<Directory"/apps/www/html/">
Options None
AllowOverrideNone
Require allgranted
</Directory>
</VirtualHost>
<VirtualHost 172.16.76.30:80>
DocumentRoot/var/www/html/
ServerNamewww.linuxinfo.top
ErrorLoglogs/bbs.error_log
Customloglogs/bbs.access_log combined #单独记录日志
<Directory"/var/www/html/">
Options None
AllowOverrideNone
Require all granted
</Directory>
</VirtualHost>
[root@node3 conf.d]# vim ../conf/httpd.conf
Listen 80
Listen 8080 #设置Httpd监听端口;
#DocumentRoot "/var/www/html" #将该行注释;
[root@node3 conf.d]# systemctl restarthttpd.service
[root@node3 ~]# mysql
MariaDB [(none)]>grantall on test.* to test@'localhost' identified by "test";
MariaDB [(none)]> grant all on test.*to test@'127.0.0.1' identified by "test";
MariaDB [(none)]> grant all on test.* totest@'172.16.76.40' identified by "test";
#此处允许172.16.76.40访问。
node4:
[root@node4 ~]# cd /etc/httpd/conf.d/
[root@node4 conf.d]# vim vhost.conf
<VirtualHost 172.16.76.40:8080>
DocumentRoot/apps/www/html/
ServerNamewww.linuxinfo.top
ErrorLoglogs/bbs-8080.error_log
Customloglogs/bbs-8080.access_log combined
<Directory"/apps/www/html/">
Options None
AllowOverrideNone
Require all granted
</Directory>
</VirtualHost>
<VirtualHost 172.16.76.40:80>
DocumentRoot/var/www/html/
ServerNamewww.linuxinfo.top
ErrorLoglogs/bbs.error_log
Customloglogs/bbs.access_log combined
<Directory"/var/www/html/">
Options None
AllowOverrideNone
Require all granted
</Directory>
</VirtualHost>
[root@node4 conf.d]# vim ../conf/httpd.conf
Listen 80
Listen 8080 #设置Httpd监听端口;
#DocumentRoot "/var/www/html" #将该行注释;
[root@node4 conf.d]# systemctl restart httpd.service
2、Discuz安装
[root@node3 ~]#cd /var/www/html/
[root@node3 ~]#unzip Discuz_X3.3_SC_UTF8_0101.zip -d html
[root@node3 ~]# chown -Rapache.apache html/
Discuz在172.16.16.30:80安装成功;因172.16.76.30还配置虚拟主机8080;
所以此处将已安装好的Discuz的html目录复制到8080虚拟主机即可;
[root@node3 ~]#cp –R /var/www/html /apps/www/html
为保证node4上static-web组网站配置相同将node3完整html目录打包传递到node4;
node3
root@node3 ~]#cd /var/www/
[root@node3 ~]#tar –cvf html.tar html/
[root@node3 ~]#scp html.tar 172.16.76.40:/var/www/
[root@node3 ~]#scp html.tar 172.16.76.40:/apps/www/
node4:
[root@node4 ~]#cd /var/www/
[root@node4 www]# tar –xvf html.tar
[root@node4 www]# cd /apps/www
[root@node4 www]# tar –xvf html.tar
注:因整个完整目录整体从172.16.76.30拷贝而来此处,所以需要修改mysql的连接地址;
[root@node4 ~]#cd /var/www/
[root@node4 upload]# vim config/config_global.php
$_config['db']['1']['dbhost'] = '172.16.76.30'; #此处设置mysql地址;
[root@node4 upload]# vim config/config_ucenter.php
define('UC_DBHOST', '172.16.76.30');
[root@node4 upload]# vim uc_server/data/config.inc.php
define('UC_DBHOST', '172.16.76.30');
即此处共计修改三个文件:
config/config_global.php
config/config_ucenter.php
uc_server/data/config.inc.php
同上: 172.16.76.40:8080 虚拟主机目录/apps/www/ 也需要修改以上三个文件。
至此LAMP全部配置完成;
写到这笔者想起来在172.16.76.30服务器上安装Discuz时其实可以配置数据库地址为172.16.76.30的这样后续复制网站目录时就不用修改那三个文件了
![1.png Haproxy+keepalivd+varnish+LAMP实验_LAMP_07](https://i-blog.csdnimg.cn/blog_migrate/9bf3592ad00dffeec73ba502744aeaea.png)
![1.png Haproxy+keepalivd+varnish+LAMP实验_LAMP_07](https://i-blog.csdnimg.cn/blog_migrate/c22c5ade3916d1cb346d9645fe1b1c71.png)
二、Haproxy配置:
node1:
[root@node1~]# cd /etc/haproxy/
root@node1 haproxy]# vim haproxy.cfg
global
log 127.0.0.1 local2 info #日志配置
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
# turn on stats unixsocket
stats socket/var/lib/haproxy/stats
defaults
mode http
log global
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive10s
timeout check 10s
maxconn 3000
frontend www
bind *:80
mode http
option forwardfor header Client-IP
log global
配置:以.php结尾所有url调度至webser服务器组即:dyanamc-web组;
以.css .js等结尾与usrl中包含/static/p_w_picpathsd等字符的请求调至webser-static
即:static-web服务器组中;
acl web url_end -i .php
acl url_static url_beg -i /static /p_w_picpaths /javascript
acl url_static url_reg -i .js.*$ .css.*$ .jpg.*$ .png.*$.gif.*$
use_backend webser if web
use_backend webser-static ifurl_static
default_backend webser
backend webser
mode http
option redispatch
option abortonclose
option forwardfor header Client-IP #将真实ip转发至后端web;http日志设置
balance source #source调度算法;会话保持(注:看最后测试)
cookie SERVERID
option httpchk GET/upload/forum.php
server web1 172.16.76.30:80 cookie server1 weight 6 check inter 2000 \
rise 2 fall 3
server web2 172.16.76.30:8080 cookie server1 weight 6 check inter 2000 \
rise 2 fall 3
backend webser-static
server web1 172.16.76.60:80 check inter2000 rise 2 fall 3
listen admin_stats
bind *:9188
mode http
stats refresh 30s
stats uri/haproxy-status #配置监控页面url
stats realm welcomelogin\ Haproxy
stats authadmin:adminpass #监控界面验证用户密码
stats hide-version
stats admin if TRUE
acl client src 172.16.250.186
block unless client
#此处配置除172.16.250.186ip之外其余ip禁止访问监控页面;
[root@node1haproxy]# systemctl restart haproxy 重启haproxy;
因Haproxy配置高可用间此配置文件拷贝至node2即可;
[root@node1 haproxy]#scp haproxy.cfg172.16.76.20:/etc/haproxy
node2:
[root@node2~]#systemctl restart haproxy
Haproxy日志输出配置
1、node1
[root@node1 ~]# vim /etc/rsyslog.d/haprxoy.conf
$Modload imudp
$UDPServerRun 514
local2.* /var/log/haproxy/haproxy_info.log
[root@node1 ~]# vim /etc/sysconfig/rsyslog
SYSLOGD_OPTIONS="-c 2 -r -m 0" #配置接收远程日志
2、node2同上
三、Keepalived配置
node1:
[root@node1 ~]# cd /etc/keepalived/
[root@node1 keepalived]# cat keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_fromkeepadmin@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id node1
vrrp_mcast_group4224.0.76.100
}
vrrp_script check_haproxy {
script "killall -0haproxy"
interval 2
weight 21
}
vrrp_instance HAporxy_VIP1 {
state MASTER # haproxy2 即node2上为BACKUP
interface eth0
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass test1111
}
virtual_ipaddress {
172.16.76.100/16 deveth0 label eth0:0
}
notify_master"/etc/keepalived/notify.sh master VIP1 vrid100"
notify_backup"/etc/keepalived/notify.sh backup VIP1 vrid100"
notify_fault"/etc/keepalived/notify.sh fault VIP1 vrid100"
track_script {
check_haproxy
}
}
vrrp_instance HAprxoy_VIP2 {
state BACKUP # haproxy2 即node2上为MASTER
interface eth0
virtual_router_id 110
priority 98
advert_int 1
authentication {
auth_type PASS
auth_pass 1111test
}
virtual_ipaddress {
172.16.76.110/16 deveth0 label eth0:1
}
notify_master"/etc/keepalived/notify.sh master VIP2 vrid110"
notify_backup"/etc/keepalived/notify.sh backup VIP2 vrid110"
notify_fault"/etc/keepalived/notify.sh fault VIP2 vrid110"
track_script {
check_haproxy
}
}
notify.sh脚本
notify.sh 脚本
#!/bin/bash
#
info="$2$3"
contact='root@localhost'
notify(){
mailsubject="$(hostname) $info tobe $1, vip floating"
mailbody="$(date +'%F %T'): vrrptransition, $(hostname) $info changed to be $1"
echo "$mailbody" | mail -s"$mailsubject" $contact
}
case$1 in
master)
notify master $2 $3
;;
backup)
notify backup $2 $3
;;
fault)
notify fault $2 $3
;;
*)
echo "Usage: $(basename$0) {master|backup|fault}"
exit 1
;;
esac
将node1Keepalive的配置文件拷贝至node2一份(注:看配置文件注释修改)
[root@node1~]# cd /etc/keepalived/
[root@node1 keepalived]# scp keepalived.conf172.16.76.20:/etc/keepalived/
node2
一次启动node1 、 node2 Keepalived
[root@node1 ~]# systemctl restart Keepalived
[root@node2 ~]# systemctl restart Keepalived
四、Varnish 配置:
node6:
[root@node6~]# cd /etc/varnish/
[root@node6 varnish]# cat default.vcl
vcl 4.0;
import directors;
probe static {
.url ="/upload/forum.php";
.interval = 5s;
.timeout = 1s;
}
backend web1 {
.host ="172.16.76.50";
.port = "8080";
.probe = static;
}
backend web2 {
.host ="172.16.76.40";
.port = "8080";
.probe = static;
}
sub vcl_init {
new webser =directors.round_robin();
webser.add_backend(web1);
webser.add_backend(web2);
}
acl purgers {
"172.16.76.30"/32;
"172.16.76.40"/32;
}
sub vcl_purge {
return(synth(200,"Purged"));
}
sub vcl_recv {
if (req.restarts == 0) {
if(req.http.x-forwarded-for) {
setreq.http.X-Forwarded-For = req.http.X-Forwarded-For +"," +client.ip;
} else {
setreq.http.X-Forwarded-For = client.ip;
}
}
set req.backend_hint =webser.backend();
if (req.url ~ "\.(html|jpg|png|bmp|jpeg|gif|js|ico|swf|css)$") {
unset req.http.cookie;
}
if (req.method == "PURGE") {
if (!client.ip ~ purgers) {
return(synth(405,"Purgingnot allowed for " + client.ip));
}
return(purge);
}
}
sub vcl_backend_response {
}
sub vcl_deliver {
if (obj.hits>0) {
set resp.http.X-Cache ="Hit Via" + " " + server.ip;
}else {
set resp.http.X-Cache ="Miss Via" + " " +server.ip;
}
}
五、修改web服务器日志配置
node3:
[root@node3~]# vim /etc/httpd/conf/httpd.conf
LogFormat"%{Client-ip}i %l %u %t \"%r\" %>s %b\"%{Referer}i\" \"%{User-Agent}i\"" combined
node4:同上
重启启动各个服务即可
六、测试
1、客户端访问测试
2、客户端访问haprxoy监控页面测试
非客户端172.16.76.186访问测试
3、varnish缓存命中查看
4、后端服务器日志查看
5、重要 假设haproxy 不设置会话保持就会出现下面的情况
验证码输入正确,但是无法登录。这是因为session会话的原因;
七、总结
本实验完成实现以下功能:
1、Haproxy负载均衡
2、Haproxy 监控页面访问控制
3、varnish缓存
4、动静分离
5、会话保持
本实验至此全部完成。实验中可能会有下瑕疵,各位注意观察出错可别怪我没有提醒哈;
若实验中真有问题欢迎指正;