LVS构架负载均衡Linux集群系统

最新推荐文章于 2024-09-21 16:02:20 发布

wonitazansa1

最新推荐文章于 2024-09-21 16:02:20 发布

阅读量1.9k

点赞数

文章标签：负载均衡 linux service tcp server networking

本文链接：https://blog.csdn.net/wonitazansa1/article/details/4680319

版权

本人初学linux，在领导督促和工作压力下，按照前人部署手册和强大搜索引擎帮助下，用lvs和mon实现的nat负载均衡，vs是suse 10.2，realserver是windows2003的IIS部署。

1 安装
1.1 网络拓扑图

1.1 检查内核是否支持lvs选项

必须在此路径下执行命令

选择Networking,如下图所示:

选择Networking options,如下图所示

选择IP:Virtual Server Configuration

如下图所示,IPVS scheduler中,包含全部的负载均衡策略(<M>为modularizes features)

1.3 安装IP虚拟服务器软件ipvsadm
Ipvsadm在安装CD1的/suse/i586目录下.需要安装为

s193000012:~ # rpm -icvh ipvsadm-1.24-123.2.i586.rpm

安装完，执行ipvsadm命令，应该有类似如下的信息出现：

s193000012:~ # ipvsadm

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

-> RemoteAddress:Port Forward Weight ActiveConn InActConn

出现类似以上信息，表明支持LVS的内核和配置工具ipvsadm已完全安装，这台Director机器已经初步安装完成，已具备构架各种方式的集群的条件。

1.4 安装mon
1.4.1 检查mon是否安装

s193000012:~ # rpm -qa |grep mon

perl-Net-Daemon-0.38-61.2

mono-data-1.2.2-12.20

smartmontools-5.37.0.20070914-5.3

mono-core-1.2.2-12.20

mono-winforms-1.2.2-12.20

dbus-1-mono-0.60-33.22

mono-web-1.2.2-12.20

openhpi-daemon-2.10.2-0.4

mon-0.99.2-368.2

    如果没有出现以上标记中内容，按照顺序安装
    perl-Time-Period-1.20-317.2.i586.rpm
    perl-Convert-BER-1.3101-206.2.i586.rpm
    perl-Mon-0.11-310.2.i586.rpm
    fping-2.4b2-15.2.i586.rpm
    mon-0.99.2-368.2.i586.rpm

1.4.2 Lvs.alert脚本
/usr/lib/mon/alert.d/lvs.alert

#!/usr/bin/perl

# lvs.alert - Linux Virtual Server alert for mon

# It can be activated by mon to remove a real server when the

# service is down, or add the server when the service is up.

use Getopt::Std;

getopts ("s:g:h:t:l:P:V:R:W:F:u");

#P:protocal

#V:virtual_server

#R:Real_server

#F:forwarding

$ipvsadm = "/sbin/ipvsadm";

$protocol = $opt_P;

$virtual_service = $opt_V;

$remote = $opt_R;

if ($opt_u) {

$weight = $opt_W;

if ($opt_F eq "nat") {

$forwarding = "-m"; }

elsif ($opt_F eq "tun") {

$forwarding = "-i"; }

else {

$forwarding = "-g"; }

if ($protocol eq "tcp") {

system("$ipvsadm -a -t $virtual_service -r $remote $forwarding -w $weight"); }

else {

system("$ipvsadm -a -u $virtual_service -r $remote $forwarding -w $weight"); } }

else {

if ($protocol eq "tcp") {

system("$ipvsadm -d -t $virtual_service -r $remote"); }

else {

system("$ipvsadm -d -u $virtual_service -r $remote"); }

};

当realserver发生变化，通过以上警告，删除或添加ipvsadm的记录

2 Mon配置文件
/etc/mon/mon.cf

# global options

cfbasedir = /etc/mon

alertdir = /usr/lib/mon/alert.d

mondir = /usr/lib/mon/mon.d

statedir = /var/lib/mon

logdir = /var/log

maxprocs = 20

histlength = 100

historicfile = mon_history.log

randstart = 60s

authtype = getpwnam

dtlogging = yes

# service telnet

#pool_ip1

hostgroup pool_ip1 10.194.3.41

watch pool_ip1

service http

interval 1s

monitor http.monitor

period wd {Mon-Sun}

alert lvs.alert -P tcp -V 10.193.0.13:80 -R 10.194.3.41:80 -W 10 -F nat

upalert lvs.alert -P tcp -V 10.193.0.13:80 -R 10.194.3.41:80 -W 10 -F nat -u 1

#pool_ip2

hostgroup pool_ip2 10.194.3.42

watch pool_ip2

service http

interval 1s

monitor http.monitor

period wd {Mon-Sun}

alert lvs.alert -P tcp -V 10.193.0.13:80 -R 10.194.3.42:80 -W 10 -F nat

upalert lvs.alert -P tcp -V 10.193.0.13:80 -R 10.194.3.42:80 -W 10 -F nat -u 1

以上是realserver发生变化，产生log，并调用lvs.alert

upalert lvs.alert -P tcp -V 10.193.0.13:80 -R 10.194.3.41:80 -W 10 -F nat -u 1
发生down 协议 vs realserver 权重负载方式恢复标志

2.1 启动mon服务

#service mon start

禁用服务

Service mon stop

重启服务

Service mon restart

2.1.1 日志

/var/log/ mon_history.log

alert pool_ip2 http 1255580913 /usr/lib/mon/alert.d/lvs.alert (-P tcp -V 10.193.0.13:80 -R 10.194.3.42:80 -W 10 -F nat) 10.194.3.42

upalert pool_ip2 http 1255580915 /usr/lib/mon/alert.d/lvs.alert (-P tcp -V 10.193.0.13:80 -R 10.194.3.42:80 -W 10 -F nat -u 1) 10.194.3.42

从上面日志里可以看出，1条日志10.194.3.42发出报警，2条日志10.194.3.42恢复正常

2.2 设置linux为路由转发模式

编译:/etc/rc.d/rc

增加: echo "1" > /proc/sys/net/ipv4/ip_forward

保存后重启，或者执行如上命令，才能生效

test -e /etc/issue-SuSE-first-run && {

cat /etc/issue-SuSE-first-run > $REDIRECT

rm -f /etc/issue-SuSE-first-run 2>/dev/null

}

echo "1" > /proc/sys/net/ipv4/ip_forward

exit 0

2.3 建立LVS可执行文件

vs_ip=10.193.0.13

vs_port=80

pool_ip1=10.194.3.41

pool_ip1_wight=10

pool_ip2=10.194.3.42

pool_ip2_wight=10

pool_port=80

#Clear all the ipvsadmin configurations

ipvsadm -C

# Choose the Weighted Round Robing

ifconfig eth0:1 down

ifconfig eth0:1 $vs_ip netmask 255.255.255.255 up

ipvsadm -A -t $vs_ip:$vs_port -s wrr

#ipvsadm -A -t $vs_ip:$vs_port -s lc

# Set Real Server

ipvsadm -a -t $vs_ip:$vs_port -r $pool_ip1:$pool_port -m -w $pool_ip1_wight

ipvsadm -a -t $vs_ip:$vs_port -r $pool_ip2:$pool_port -m -w $pool_ip2_wight

#Display the current configurations

ipvsadm

以上是建立lvs可执行文件，可以放在任意目录，我保存在/etc/init.d下
需要执行lvs
当前目录>./lvs

3 测试

启动：
Service mon restart（前面已经执行过）
echo "1" > /proc/sys/net/ipv4/ip_forward（手工执行，或者重启服务器）
当前目录>./lvs（前面已经执行过）

s193000012:/etc/init.d # ipvsadm //显示内容

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

-> RemoteAddress:Port Forward Weight ActiveConn InActConn

TCP 10.193.0.13:http wrr

-> 10.194.3.42:http Masq 10 0 1

-> 10.194.3.41:http Masq 10 1 0

s193000012:/etc/init.d # ipvsadm –ln //显示端口

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

-> RemoteAddress:Port Forward Weight ActiveConn InActConn

TCP 10.193.0.13:80 wrr

-> 10.194.3.42:80 Masq 10 0 1

-> 10.194.3.41:80 Masq 10 1 0

s193000012:/etc/init.d # ipvsadm –lnc //显示当前连接数

IPVS connection entries

pro expire state source virtual destination

TCP 00:46 TIME_WAIT 10.0.100.222:1938 10.193.0.13:80 10.194.3.42:80

TCP 01:51 TIME_WAIT 10.0.100.222:1963 10.193.0.13:80 10.194.3.41:80

如果现在断开10.193.3.41

在/var/log/ mon_history.log

alert pool_ip1 http 1255580551 /usr/lib/mon/alert.d/lvs.alert (-P tcp -V 10.193.0.13:80 -R 10.194.3.41:80 -W 10 -F nat) 10.194.3.41

alert pool_ip1 http 1255580553 /usr/lib/mon/alert.d/lvs.alert (-P tcp -V 10.193.0.13:80 -R 10.194.3.41:80 -W 10 -F nat) 10.194.3.41

执行ipvsadm，发现少了10.193.3.41，这样就不会发生访问异常

s193000012:/etc/init.d # ipvsadm

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

-> RemoteAddress:Port Forward Weight ActiveConn InActConn

TCP 10.193.0.13:http wrr

-> 10.194.3.42:http Masq 10 0 0

恢复10.193.3.41的连接，发现upalert日志，说明已经恢复

alert pool_ip1 http 1255596062 /usr/lib/mon/alert.d/lvs.alert (-P tcp -V 10.193.0.13:80 -R 10.194.3.41:80 -W 10 -F nat) 10.194.3.41

upalert pool_ip1 http 1255596064 /usr/lib/mon/alert.d/lvs.alert (-P tcp -V 10.193.0.13:80 -R 10.194.3.41:80 -W 10 -F nat -u 1) 10.194.3.41

执行ipvsadm，10.193.3.41记录已经到ipvsadm中了

s193000012:/etc/init.d # ipvsadm

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

-> RemoteAddress:Port Forward Weight ActiveConn InActConn

TCP 10.193.0.13:http wrr

-> 10.194.3.41:http Masq 10 0 0

-> 10.194.3.42:http Masq 10 0 0

s193000012:/etc/init.d #

4 问题
1、realerver断开后，mon_ history.log，系统不断有警报日志弹出，执行ipvsadm，发现报警的realerver还在ipvsadm中，linux弹出

/usr/lib/mon/alert.d/lvs.alert: line 15: use: command not found

/usr/lib/mon/alert.d/lvs.alert: line 16: syntax error near unexpected token `"s:g:h:t:l:P:V:R:W:F:u"'

/usr/lib/mon/alert.d/lvs.alert: line 16: `getopts ("s:g:h:t:l:P:V:R:W:F:u");'

问题出在usr/lib/mon/alert.d/lvs.alert的15、16行，上网查了半天，发现可能是编码问题，我是从剪切板粘贴过来的，从alert.template复制一个lvs.alert，将use Getopt::Std;
getopts ("s:g:h:t:l:P:V:R:W:F:u");按照所需进行修改，其他内容进行粘贴，问题解决！