linux下利用Haproxy和keepalived实现简单负载均衡

架构拓扑图如下：



haproxy
VIP：  192.168.1.150
Master：192.168.1.10
Slave： 192.168.1.11
之间用keepalived实现HA
webserver1：192.168.1.101
webserver2：192.168.1.102
webserver3：192.168.1.103
webserver4：192.168.1.104

1.下载：

Quotation

#wget http://haproxy.1wt.eu/download/1.3/src/haproxy-1.3.14.4.tar.gz

2.解压安装

Quotation

#tar  –zxvf  haproxy-1.3.14.4.tar.gz
#tar zxvf haproxy-1.3.14.4..tar.gz
#mv  haproxy-1.3.14.4  haproxy
#cd haproxy
#make TARGET=linux26  

（2.4内核版本 make TARGET=linux24
freebsd  make  TARGET=freebsd
solaris系统  make TARGET=solaris CPU=ultrasparc REGEX=static-pcre）
然后增加一个haproxy用户

3.配置haproxy

 

3. 复制并修改相应的实例文件：

mkdir /etc/haproxy
cp /usr/local/haproxy/examples/haproxy.cfg /etc/haproxy/
cp /usr/local/haproxy/examples/haproxy.init /etc/init.d/haproxy
chmod a+x /etc/init.d/haproxy

vi /etc/haproxy/haproxy.cfg

Quotation

global
        chroot /var/chroot/haproxy
        daemon
        gid     501
        log 127.0.0.1 local3
        nbproc  2
        pidfile /var/run/haproxy-private.pid

        ulimit-n        65535
        user    haproxy
        maxconn         32000
        spread-checks           4
        tune.maxaccept          8
        tune.maxpollevents      100


defaults askwan
        log     global
        mode    http
        option  httplog
        option  dontlognull
        log 127.0.0.1 local3
        retries 3
        option redispatch
        maxconn 32000
        contimeout      5000
        clitimeout      50000
        srvtimeout      50000


listen  askwan  0.0.0.0:80
        appsession JSESSIONID len 52 timeout 3h
        cookie SRV insert indirect nocache
        mode http
        stats enable
        stats hide-version
        stats uri /haproxy-stats
        stats realm Haproxy/ statistics
        stats auth askwan:askwan
        stats refresh 3s
        monitor-uri /haproxy_test
        balance roundrobin
        option httpclose
        option forwardfor
        option httpchk HEAD /index.html HTTP/1.0  #（访问index.html来判断服务器是否健康，所以要保证每个服务器它都是存在且可以被访问的！否则被视为分发服务器当机）

        server server1 192.168.1.101:80 check inter 2000  
        server server2 192.168.1.102:80 check inter 2000
        server server3 192.168.1.103:80 check inter 2000
        server server4 192.168.1.104:80 check inter 2000

启动haproxy服务 ： ./haproxy -d -f /etc/haproxy/haproxy.cfg （注意：80端口不要被占用否则就悲剧了 T _ T）

 

下面是监控页面   http://服务器地址/haproxy-stats




附上haproxy启动脚本：

Quotation

#!/bin/sh
#
# chkconfig: - 85 15
# description: HA-Proxy is a TCP/HTTP reverse proxy which is particularly suited /
#              for high availability environments.
# processname: haproxy
# config: /etc/haproxy/haproxy.cfg
# pidfile: /var/run/haproxy.pid

# Script Author: Simon Matter
# Version: 2004060600

# Source function library.
if [ -f /etc/init.d/functions ]; then
  . /etc/init.d/functions
elif [ -f /etc/rc.d/init.d/functions ] ; then
  . /etc/rc.d/init.d/functions
else
  exit 0
fi

# Source networking configuration.
. /etc/sysconfig/network

# Check that networking is up.
[ ${NETWORKING} = "no" ] && exit 0

# This is our service name
BASENAME=`basename $0`
if [ -L $0 ]; then
  BASENAME=`find $0 -name $BASENAME -printf %l`
  BASENAME=`basename $BASENAME`
fi

[ -f /etc/$BASENAME/$BASENAME.cfg ] || exit 1

RETVAL=0

start() {
  /usr/sbin/$BASENAME -c -q -f /etc/$BASENAME/$BASENAME.cfg
  if [ $? -ne 0 ]; then
    echo "Errors found in configuration file, check it with '$BASENAME check'."
    return 1
  fi

  echo -n "Starting $BASENAME: "
  daemon /usr/sbin/$BASENAME -D -f /etc/$BASENAME/$BASENAME.cfg -p /var/run/$BASENAME.pid
  RETVAL=$?
  echo
  [ $RETVAL -eq 0 ] && touch /var/lock/subsys/$BASENAME
  return $RETVAL
}

stop() {
  echo -n "Shutting down $BASENAME: "
  killproc $BASENAME -USR1
  RETVAL=$?
  echo
  [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/$BASENAME
  [ $RETVAL -eq 0 ] && rm -f /var/run/$BASENAME.pid
  return $RETVAL
}

restart() {
  /usr/sbin/$BASENAME -c -q -f /etc/$BASENAME/$BASENAME.cfg
  if [ $? -ne 0 ]; then
    echo "Errors found in configuration file, check it with '$BASENAME check'."
    return 1
  fi
  stop
  start
}

check() {
  /usr/sbin/$BASENAME -c -q -V -f /etc/$BASENAME/$BASENAME.cfg
}

rhstatus() {
  status $BASENAME
}

condrestart() {
  [ -e /var/lock/subsys/$BASENAME ] && restart || :
}

# See how we were called.
case "$1" in
  start)
    start
    ;;
  stop)
    stop
    ;;
  restart)
    restart
    ;;
  reload)
    restart
    ;;
  condrestart)
    condrestart
    ;;
  status)
    rhstatus
    ;;
  check)
    check
    ;;
  *)
    echo $"Usage: $BASENAME {start|stop|restart|reload|condrestart|status|check}"
    exit 1
esac

exit $?

4.在两台haproxy机器上安装配置好keepalived
Keepalived配置
VIP 192.168.1.150
Realserver  192.168.1.10 （MASTER）
RealServer  192.168.1.11 （SLAVER）

分别在两台机器上配置keepalived.conf文件

Quotation

global_defs {
   notification_email {
     askwan@yahoo.com
   }
   notification_email_from askwan@askwan.com
   smtp_server 192.168.1.3
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass askwan
    }
    virtual_ipaddress {
        192.168.1.150
    }

5调试

这个是Master机keepalived调试信息：

Quotation

Apr 22 08:32:22 db1 Keepalived: Starting Keepalived v1.1.15 (04/22,2008)
Apr 22 08:32:22 db1 Keepalived: daemon is already running
Apr 22 08:32:24 db1 Keepalived: Starting Keepalived v1.1.15 (04/22,2008)
Apr 22 08:32:24 db1 Keepalived: daemon is already running
Apr 22 08:32:46 db1 Keepalived: Starting Keepalived v1.1.15 (04/22,2008)
Apr 22 08:32:46 db1 Keepalived: daemon is already running
Apr 22 08:37:42 db1 Keepalived: Starting Keepalived v1.1.15 (04/22,2008)
Apr 22 08:37:42 db1 Keepalived: daemon is already running
Apr 22 08:39:34 db1 Keepalived: Starting Keepalived v1.1.15 (04/22,2008)
Apr 22 08:39:34 db1 Keepalived: daemon is already running
Apr 22 08:41:20 db1 sshd(pam_unix)[6317]: session opened for user root by root(uid=0)
Apr 22 08:41:32 db1 Keepalived: Terminating on signal
Apr 22 08:41:32 db1 Keepalived_vrrp: Terminating VRRP child process on signal
Apr 22 08:41:32 db1 Keepalived: Stopping Keepalived v1.1.15 (04/22,2008)
Apr 22 08:42:03 db1 Keepalived: Starting Keepalived v1.1.15 (04/22,2008)
Apr 22 08:42:03 db1 Keepalived_vrrp: Using MII-BMSR NIC polling thread...
Apr 22 08:42:03 db1 Keepalived_vrrp: Registering Kernel netlink reflector
Apr 22 08:42:03 db1 Keepalived_vrrp: Registering Kernel netlink command channel
Apr 22 08:42:03 db1 Keepalived_vrrp: Registering gratutious ARP shared channel
Apr 22 08:42:03 db1 Keepalived: Starting VRRP child process, pid=6358
Apr 22 08:42:03 db1 Keepalived_vrrp: Opening file '/usr/local/keepalived/etc/keepalived/keepalived.conf'.
Apr 22 08:42:03 db1 Keepalived_vrrp: Configuration is using : 33290 Bytes
Apr 22 08:42:03 db1 Keepalived_vrrp: VRRP sockpool: [ifindex(2), proto(112), fd(7,8)]
Apr 22 08:42:04 db1 Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATE
Apr 22 08:42:05 db1 Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATE
Apr 22 08:42:05 db1 Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.
Apr 22 08:42:05 db1 Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.150
Apr 22 08:42:10 db1 Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.150

可以看到此机此时处于MASTER状态

slave机调试信息

Quotation

Apr 20 08:43:28 db2 Keepalived: Starting Keepalived v1.1.15 (04/22,2008)
Apr 22 08:43:28 db2 Keepalived_vrrp: Using MII-BMSR NIC polling thread...
Apr 22 08:43:28 db2 Keepalived_vrrp: Registering Kernel netlink reflector
Apr 22 08:43:28 db2 Keepalived_vrrp: Registering Kernel netlink command channel
Apr 22 08:43:28 db2 Keepalived_vrrp: Registering gratutious ARP shared channel
Apr 22 08:43:28 db2 Keepalived: Starting VRRP child process, pid=5857
Apr 22 08:43:28 db2 Keepalived_vrrp: Opening file '/usr/local/keepalived/etc/keepalived/keepalived.conf'.
Apr 22 08:43:28 db2 Keepalived_vrrp: Configuration is using : 36052 Bytes
Apr 22 08:43:28 db2 Keepalived_vrrp: VRRP_Instance(VI_1) Entering BACKUP STATE
Apr 22 08:43:28 db2 Keepalived_vrrp: VRRP sockpool: [ifindex(2), proto(112), fd(7,8)]

可以看到此机此时处于BACKUP状态

5.测试

地址已经分配完成后，可以开始做测试 测试过程简单 省略了
在HA的测试过程说一下
192.168.1.10宕机后192.168.1.11接管
192.168.1.10恢复后 192.168.1.11移交服务权限10
本方案经过我多次测试基本达到了架构拓扑图中的需求。这样一个简单高可用集群方案搭建完成。


更过信息请参阅：
http://haproxy.1wt.eu/
http://www.keepalived.org /

 

 

------------------------------------------------------------------------------------------------------------

 

优化：
ulimit –n #设置最大同时打开文件数（控制最大socket连接数等），非常重要
vim /etc/security/limits.conf

• soft nofile 150000
• hard nofile 150000

#pam_limits.so模块就会从/etc/security/limits.conf文件中读取配置来设置这些限制值
Vim /etc/pam.d/login
session required /lib/security/pam_limits.sos