HAproxy配置

转载自：http://www.360doc.com/content/17/0328/19/41512315_640903721.shtml

本文是我给公司内部写的一个简单的配置文档，文中只有配置步骤，省掉了原理说明部分。

polygun2000原创，转载请注明: 来源于polygun2000博客 http://blog.sina.com.cn/polygun2000

一、功能需求

1.四层负载均衡(TCP)和七层负载均衡(HTTP)

2.会话保持

3.IP地址透传

二、系统结构

三、系统组件介绍

haproxy:   http://haproxy.1wt.eu

1.基于 TCP 和 HTTP 协议的高效能负载均衡器(不同于nginx,haproxy本身不具有web server功能)。

2.基于GPL协议,开源软件。

3.高效,稳定,安全性高,适合重负载使用,支持10GE网卡。

4.负载均衡算法灵活: 轮询，静态轮询,最小连接数，源地址hash，基于url等。

5.支持透明代理,限速等高级功能。

tproxy: http://www.balabit.com/support/community/products/tproxy

1.支持透明代理的内核补丁,自2.6.28以后已经进入主线内核。

2.结合haproxy可以使用户IP地址透传给后端服务器。

keepalived: http://www.keepalived.org

1.用来防止路由器出现单点故障的热备份软件，最早用于与LVS结合。

2.使用VRRP协议。

四、配置过程简述

 

五、具体配置步骤

1.环境准备

 

硬件选择:  E5-2600CPU+Intel服务器网卡

操作系统: 最小化安装CentOS 6.3 x86_64

 

a.关闭网卡中断调节
[root@ modprobe.d]# vi /etc/modprobe.d/intel-nic.conf
options igb InterruptThrottleRate=0,0,0,0
或者
options ixgbe InterruptThrottleRate=0,0
 

b.设置网卡中断CPU亲和

set_irq_affinity.sh脚本包含在Intel官方的ixgbe驱动中，下载地址:
https://downloadcenter.intel.com/download/14687/Network-Adapter-Driver-for-PCI-E-10-Gigabit-Network-Connections-under-Linux-

 

安装163,epel源

[root@haproxy ~]# yum install wget

[root@haproxy ~]# wget http://mirrors.163.com/.help/CentOS6-Base-163.repo

[root@haproxy ~]# wget http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm

[root@haproxy ~]# mv CentOS6-Base-163.repo /etc/yum.repos.d/CentOS-Base.repo

[root@haproxy ~]# rpm -ivh epel-release-6-8.noarch.rpm

[root@haproxy ~]# yum update

2.编译安装pcre

[root@haproxy ~]# yum install gcc gcc-c++ make zlib-devel bzip2-devel

[root@haproxy ~]# wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.32.tar.bz2

[root@haproxy ~]# tar xvjf pcre-8.32.tar.bz2

[root@haproxy ~]# ./configure --prefix=/usr \

--docdir=/usr/share/doc/pcre-8.32 \

--enable-utf --enable-unicode-properties \

--enable-pcregrep-libz --enable-pcregrep-libbz2

[root@haproxy ~]# make

[root@haproxy ~]# make check

[root@haproxy ~]# make install

 

3.编译安装haproxy

[root@haproxy ~]# yum install openssl-devel

[root@haproxy ~]# wget http://haproxy.1wt.eu/download/1.5/src/devel/haproxy-1.5-dev17.tar.gz

[root@haproxy ~]# tar xvzf haproxy-1.5-dev17.tar.gz

[root@haproxy ~]# cd haproxy-1.5-dev17

[root@haproxy ~]# make TARGET=linux26 USE_STATIC_PCRE=1 \

USE_REGPARM=1 USE_LINUX_TPROXY=1 USE_OPENSSL=1 USE_ZLIB=1 ARCH=x86_64

[root@haproxy ~]# make install

 

4.创建haproxy启动脚本

来源: http://mattiasgeniar.be/2010/11/04/a-custom-init-d-start-up-script-for-haproxy-start-stop-restart-reload-checkconfig/

直接下载连接: http://mattiasgeniar.be/downloads/haproxy/haproxy.init

[root@haproxy ~]# vi /etc/init.d/haproxy

#----------------------------

#!/bin/sh

#

# custom haproxy init.d script, by Mattias Geniar

#

# haproxy         starting and stopping the haproxy load balancer

#

# chkconfig: 345 55 45

# description: haproxy is a TCP loadbalancer

# probe: true

 

# Source function library.

. /etc/rc.d/init.d/functions

 

# Source networking configuration.

. /etc/sysconfig/network

 

# Check that networking is up.

[ ${NETWORKING} = "no" ] && exit 0

 

[ -f /usr/local/sbin/haproxy ] || exit 0

 

[ -f /etc/haproxy/haproxy.conf ] || exit 0

 

# Define our actions

checkconfig() {

        # Check the config file for errors

        /usr/local/sbin/haproxy -c -q -f /etc/haproxy/haproxy.conf

        if [ $? -ne 0 ]; then

                 echo "Errors found in configuration file."

                return 1

        fi

 

        # We're OK!

        return 0

}

 

start() {

        # Check config

        /usr/local/sbin/haproxy -c -q -f /etc/haproxy/haproxy.conf

        if [ $? -ne 0 ]; then

                echo "Errors found in configuration file."

                return 1

        fi

 

        echo -n "Starting HAProxy: "

        daemon /usr/local/sbin/haproxy -D -f /etc/haproxy/haproxy.conf -p /var/run/haproxy.pid

 

        RETVAL=$?

        echo

        [ $RETVAL -eq 0 ] && touch /var/lock/subsys/haproxy

        return $RETVAL

}

 

stop() {

        echo -n "Shutting down HAProxy: "

        killproc haproxy -USR1

 

        RETVAL=$?

        echo

        [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/haproxy

        [ $RETVAL -eq 0 ] && rm -f /var/run/haproxy.pid

        return $RETVAL

}

 

restart() {

        /usr/local/sbin/haproxy -c -q -f /etc/haproxy/haproxy.conf

        if [ $? -ne 0 ]; then

                echo "Errors found in configuration file."

                return 1

        fi

 

        stop

        start

}

 

check() {

        /usr/local/sbin/haproxy -c -q -V -f /etc/haproxy/haproxy.conf

}

 

rhstatus() {

        status haproxy

}

 

reload() {

        /usr/local/sbin/haproxy -c -q -f /etc/haproxy/haproxy.conf

        if [ $? -ne 0 ]; then

                echo "Errors found in configuration file."

                return 1

        fi

 

        echo -n "Reloading HAProxy config: "

        /usr/local/sbin/haproxy -f /etc/haproxy/haproxy.conf -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)

 

        success $"Reloading HAProxy config: "

        echo

}

 

 

# Possible parameters

case "$1" in

  start)

        start

        ;;

  stop)

        stop

        ;;

  status)

        rhstatus

        ;;

  restart)

        restart

        ;;

  reload)

        reload

        ;;

  checkconfig)

        check

        ;;

  *)

        echo "Usage: haproxy {start|stop|status|restart|reload|checkconfig}"

        exit 1

esac

 

exit 0

#----------------------------

 

[root@haproxy ~]# chmod +x /etc/init.d/haproxy

 

设置开机启动haproxy服务

[root@haproxy ~]# chkconfig --add haproxy

[root@haproxy ~]# chkconfig haproxy on

5.配置haproxy

 

创建chroot目录,确保该目录为空,且其帐号不可访问。

[root@haproxy ~]# mkdir /var/haproxy

[root@haproxy ~]# chmod o= /var/haproxy

 

创建haproxy配置文件

[root@haproxy ~]# mkdir /etc/haproxy

[root@haproxy ~]# vi /etc/haproxy/haproxy.conf

global段配置

#全局配置

global

    maxconn 32768 # Max simultaneous connections from an upstream server

    spread-checks 5 # Distribute health checks with some randomness

    chroot /var/haproxy

    daemon

    log 127.0.0.1 local0

    log 127.0.0.1 local1 notice

    #debug # Uncomment for verbose logging

defaults段配置

#默认配置,应用于所有下边的服务

defaults

    log global

    mode http

    balance roundrobin

    retries 3

    option abortonclose # abort request if client closes output channel while waiting

    option httpclose # add "Connection:close" header if it is missing

    option forwardfor # insert x-forwarded-for header so that app servers can see both proxy and client IPs

    option redispatch # any server can handle any session

    option httplog

    option dontlognull

    timeout http-request 5s    #aginst Slowloris attack

    timeout client 60s

    timeout connect 9s

    timeout server 30s

    timeout check 5s

    stats enable

    errorfile 503 /etc/haproxy/errors/503.http

stat监控配置

#配置haproxy的状态监控

listen stats

    bind 192.168.10.132:8888

    stats uri /

    stats realm Haproxy\ Statistics

    stats auth hadmin:yhXV2WAbybXd1euzEXbe

    stats refresh 20

log配置

1.配置rsyslog以接收haproxy日志

[root@haproxy ~]# vi /etc/rsyslog.d/haproxy.conf

 

# Custom log facilities for haproxy

local0.* -/var/log/haproxy0a.log

local1.* -/var/log/haproxy1a.log

 

$ModLoad imudp

# load the imudp module for rsyslog

# provides UDP syslog reception

 

# start UDP server on this port, "*" means all addresses

$UDPServerRun 514

 

# local IP address (or name) the UDP listens should bind to

$UDPServerAddress 127.0.0.1

 

[root@haproxy ~]# /etc/init.d/rsyslog restart

 

注释:

/var/log/haproxy0a.log前边的"-"减号意味着取消日志同步写入。

这可以优化一下磁盘写入,尤其是在非常繁忙的系统中。

不过如果突然断电,可能会损失一些未写入硬盘的日志。

 

2.配置logrotate

[root@haproxy ~]# vi /etc/logrotate.d/haproxy

 

/var/log/haproxy*.log

{

    daily

    rotate 4

    missingok

    notifempty

    compress

    delaycompress

    sharedscripts

    postrotate

        /etc/init.d/haproxy reload >/dev/null

    endscript

}

 

注释:

如果站点数量较多,可能会希望将不同站点的日志分开,可以看看后边的"参考文档E"。

http应用配置

listen VIP_64.4.2.111

   bind 64.4.2.111:80

   cookie SERVERID insert indirect nocache

   server s31 192.168.10.31:80 check cookie s1

   server s32 192.168.10.32:80 check cookie s2

 

tcp应用配置

listen VIP_64.4.2.118

   bind 64.4.2.118:22186

   mode tcp

   option tcplog

   server s41 192.168.10.41:22186 check

   server s42 192.168.10.42:22186 check

 

会话保持配置

#需要做会话保持的tcp配置,采用源地址hash

listen VIP_64.4.2.109

    bind 64.4.2.109:1235

    balance source

    option tcplog

    hash-type consistent # optional

    server s11 192.168.10.11:1235 check

    server s12 192.168.10.12:1235 check

 #需要做会话保持的http配置

listen VIP_64.4.2.111

   bind 64.4.2.111:80

   cookie SERVERID insert indirect nocache

   server s31 192.168.10.31:80 check cookie s1

   server s32 192.168.10.32:80 check cookie s2

 

源地址透传配置

#需要查看用户真实IP的配置

listen VIP_64.4.2.118

   bind 64.4.2.118:22186

   mode tcp

   option tcplog

   source 0.0.0.0 usesrc clientip

   server s41 192.168.10.41:22186 check

   server s42 192.168.10.42:22186 check

 

为TPROXY设置iptables规则

[root@haproxy ~]# /sbin/iptables -t mangle -N DIVERT

[root@haproxy ~]# /sbin/iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT

[root@haproxy ~]# /sbin/iptables -t mangle -A DIVERT -j MARK --set-mark 1

[root@haproxy ~]# /sbin/iptables -t mangle -A DIVERT -j ACCEPT

[root@haproxy ~]# /sbin/ip rule add fwmark 1 lookup 100

[root@haproxy ~]# /sbin/ip route add local 0.0.0.0/0 dev lo table 100

 

 #给tproxy后端做NAT

[root@haproxy ~]# /sbin/iptables -t nat -A POSTROUTING -s backend's_ip -o eht0 -j MASQUERADE

 

在后端服务器上设置haproxy为默认网关

[root@backend ~]# ip route add default via haproxy_lanip

5.相关内核参数调整

[root@haproxy ~]# vi /etc/sysctl.conf

 #允许ip转发

net.ipv4.ip_forward = 1

 

#设置松散逆向路径过滤

net.ipv4.conf.default.rp_filter = 2

net.ipv4.conf.all.rp_filter = 2

net.ipv4.conf.eth0.rp_filter = 0

 

#允许ICMP重定向

net.ipv4.conf.all.send_redirects = 1

net.ipv4.conf.default.send_redirects = 1

 

#发送到一个监听的socket上的最大已完成连接队列长度

#三次握手已经完成,但还未被应用层接收(accept),但也处于ESTABLISHED状态

#队列长度由listen的backlog参数和内核的 net.core.somaxconn 参数共同决定

#当这个队列满了之后,不管未完成连接队列是否已满,是否启用syncookie,都不在接收新的SYN请求.

net.core.somaxconn = 32768

 

#允许绑定到非本地地址,用于keepalived

net.ipv4.ip_nonlocal_bind = 1

 

#增加可用的端口范围

net.ipv4.ip_local_port_range = 1024 65023

 

#防攻击使用,如无必要一定要设置成0

net.ipv4.tcp_abort_on_overflow = 0

 

#如果套接字由本端要求关闭,这个参数决定了它保持在FIN-WAIT-2状态的时间,缺省值是60秒。

#减小这个值,可以使TCP/IP更快的释放连接,腾出更多资源给新连接。推荐15-30秒。

net.ipv4.tcp_fin_timeout = 10

 

#最后一个数据包发送完成和第一个keepalive包被检测到之间的时间间隔

#表示当keepalive起用的时候,TCP发送keepalive消息的频度,缺省是2小时。

net.ipv4.tcp_keepalive_time = 300

 

#系统所能处理不属于任何进程的TCP sockets最大数量。

#假如超过这个数量,那么不属于任何进程的连接会被立即reset,并同时显示警告信息。

net.ipv4.tcp_max_orphans = 262144

 

#backlog队列是一个大的内存结构,用来处理收到的带有SYN标记的数据包,直到三次握手完成。

#这个参数控制了同一时间内操作系统可以处理多少个半开连接,当连接数达到这个数值的设定后,系统会丢弃随后的请求。

net.ipv4.tcp_max_syn_backlog = 16384

 

#表示系统同时保持TIME_WAIT套接字的最大数量,如果超过这个数字,TIME_WAIT套接字将立刻被清除并打印警告信息。

net.ipv4.tcp_max_tw_buckets = 262144

 

#对于远端的连接请求SYN,内核会发送SYN ＋ ACK数据报,以确认收到上一个 SYN连接请求包。

#这是所谓的三次握手( threeway handshake)机制的第二个步骤。这里决定内核在放弃连接之前所送出的 #SYN+ACK数目。如果你的网站SYN_RECV状态确实挺多,为了避免syn攻击,那么可以调节重发的次数。

net.ipv4.tcp_synack_retries = 3

 

#开启/关闭SYN Cookies

#当启动SYN Cookie时,主机在发送 SYN/ACK 确认封包前,会要求 Client 端在短时间内回复一个序号

#这个序号包含许多原本 SYN 封包内的信息,包括 IP、port 等。

#若 Client 端可以回复正确的序号,那么主机就确定该封包为可信的,因此会发送 SYN/ACK 封包,否则就不理会此一封包。

#这个参数不会提高性能,而且违背TCP协议,如果不是遭到SYN Flood攻击,不要打开。

net.ipv4.tcp_syncookies = 0

 

#根据RFC1323,会向TCP包头中插入12byte,2.6内核的Linux默认是打开的,某些情况下timestamp数值有可能溢出造成TCP超时

#建议关闭。

net.ipv4.tcp_timestamps = 0

 

#开启TCP连接中TIME-WAIT sockets的快速回收

net.ipv4.tcp_tw_recycle = 1

 

#开启重用,允许将TIME-WAIT sockets重新用于新的TCP连接

net.ipv4.tcp_tw_reuse = 1

 

#如果TCP窗口大小超过65536,需要此选项打开大TCP窗口支持。

net.ipv4.tcp_window_scaling=1

 

#决定TCP协议栈如何使用内存,单位是内存分页,而不是字节。每个内存分页一般为4K。

#当超过第二个值时,TCP进入pressure模式,此时TCP尝试稳定其内存的使用,

#当小于第一个值时,就退出pressure模式,TCP不会考虑释放内存。

#当内存占用超过第三个值时,TCP就拒绝分配socket了,查看dmesg,会打出很多的日志“TCP: too many of orphaned sockets”。

#如果不是非常必要,一般不要动系统默认的值,默认值一般来说够用了

net.ipv4.tcp_mem = "786432 2097152 3145728"

 

#TCP流中重排序的数据包最大数量

net.ipv4.tcp_reordering = 3

 

#系统auto-tuning时,每个socket使用的内存。分别是最小,缺省,最大TCP接收窗口的内存大小,单位byte

#如果设置net.core.rmem_default,则该值会覆盖缺省值

#如果设置net.core.rmem_max,则该值会覆盖最大值

net.ipv4.tcp_rmem = "4096 87380 16777216"

6.keepalived配置

 

安装keepalived

[root@haproxy ~]# yum install keepalived

 

配置keepalived

[root@haproxy ~]# vi /etc/keepalived/keepalived.conf

 

    ! Configuration File for keepalived 

     

    global_defs {  # global_defs全局配置标识,说明这个区域{}是全局配置

       notification_email {    # 发送email通知,以及email发送给哪些邮件地址,邮件地址可以多个,每行一个。

            admin@demo.com 

       } 

       notification_email_from admin@demo.com  # 发送通知邮件时邮件源地址是谁

       smtp_connect_timeout 3  # smtp连接超时时间

       smtp_server 127.0.0.1   # 发送email时使用的smtp服务器地址

       router_id haproxy_101   # 机器标识,从节点为haproxy_102

    } 

    vrrp_script chk_haproxy {  # 定义脚本名字

        script "killall -0 haproxy" 

        interval 2    # 脚本执行间隔2s

        weight 10  # 脚本结果导致的优先级变更：10表示优先级+10；-10则表示优先级-10

           fall 2    # require 2 failures for KO
           rise 2   # require 2 successes for OK

    } 

    vrrp_instance VI_1 {  # vrrp实例名称

        interface eth1  # 实例绑定的网卡,因为在配置虚拟IP的时候必须是在已有的网卡上添加的

        state MASTER # 从节点则此此处为BACKUP ,需要大写这些单词

        priority 101 # 设置本节点的优先级,数值愈大,优先级越高,优先级高的为master 

        virtual_router_id 50 # 主、备机的virtual_router_id必须相同！！ 

        garp_master_delay 1 # 主从切换时间,单位为秒。 

     

        authentication {  # 设置认证,同一vrrp实例MASTER与BACKUP 使用相同的密码才能正常通信。

            auth_type PASS  # 认证方式,可以是PASS或AH两种认证方式

            auth_pass U5vXgwcveTuDt66MxJa7  # 认证密码

        } 

 

        virtual_ipaddress {  # 这里设置的就是VIP，也就是用工作的虚拟IP地址,VIP最多20个

           64.4.2.110/24 dev eth0

        }

        virtual_ipaddress_excluded {  # 超过20个VIP可以添加在virtual_ipaddress_excluded中,这些VIP不需要发送检测包

           64.4.2.111/24 dev eth0

           64.4.2.112/24 dev eth0

           202.113.58.7/24 dev eth1

        }

                  

        track_interface {  # 跟踪接口,设置额外的监控,里面任意一块网卡出现问题,都会进入故障(FAULT)状态

           eth0 

           eth1 

        } 

                  

        track_script {  # 引用vrrp_script,有点类似脚本里面的函数引用一样,先定义,后引用函数名

            chk_haproxy  # 调用脚本必须放在virtual_ipaddress之后

        } 

     

        #状态通知 

        notify_master /etc/keepalived/scripts/be_master.sh   # 当进入Master状态时会呼叫notify_master

        notify_backup /etc/keepalived/scripts/be_backup.sh   # 当进入Backup状态时会呼叫notify_backup

        notify_fault  /etc/keepalived/scripts/be_fault.sh    # 当发现异常情况时进入Fault状态呼叫notify_fault

        notify_stop   /etc/keepalived/scripts/be_stop.sh     # 当Keepalived程序终止时则呼叫notify_stop

    } 

 

确认keepalived工作正常

[root@haproxy ~]# tcpdump -v -i eth0 host 224.0.0.18

tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
16:54:01.743275 IP (tos 0x0, ttl 255, id 451, offset 0, flags [none], proto: VRRP (112), length: 96) 10.10.28.5 > 224.0.0.18: VRRPv2, Advertisement, vrid 51, prio 103, authtype simple, intvl 1s, length 76, addrs(15): 123.12.15.2,123.12.15.3[|vrrp]
16:54:02.744241 IP (tos 0x0, ttl 255, id 452, offset 0, flags [none], proto: VRRP (112), length: 96) 10.10.28.5 > 224.0.0.18: VRRPv2, Advertisement, vrid 51, prio 103, authtype simple, intvl 1s, length 76, addrs(15): 123.12.15.2,123.12.15.3[|vrrp]

 

          10.10.28.5 - your eth0 ip.
          123.12.15.2 and 123.12.15.3 - Virtual IPs manage by keepalived.
          224.0.0.18 - multicast request.

    在某些网络环境下,可能不能够使用multicast来检测keepalived的心跳，所以需要使用unicast来检测，只需要在vrrp_instance配置段中加入如下:

 

    unicast_src_ip 10.188.100.20   # 指定使用unicast，后跟keepalived监听的接口IP

   unicast_peer {            # 指定另一个keepalived节点监听的IP地址 

     10.188.100.21

    }

 

另外keepalived可以很好的支持VLAN，所以在上述的配置中，所有涉及dev eth0这样的部分，都可以是类似eth0.188这样的VLAN接口。这个可以很好的应用于单接口，多VLAN的环境下。

 

六、进阶应用

1.限制单个IP的并发连接数

frontend ft_web

    bind 0.0.0.0:8080

 

    # Table definition 

    stick-table type ip size 100k expire 30s store conn_cur

 

    # Allow clean known IPs to bypass the filter

    tcp-request connection accept if { src -f /etc/haproxy/whitelist.lst }

    # Shut the new connection as long as the client has already 10 opened

    tcp-request connection reject if { src_conn_cur ge 10 }

    tcp-request connection track-sc1 src

  

2.限制单个IP建立连接的频率

frontend ft_web

    bind 0.0.0.0:8080

 

    # Table definition 

    stick-table type ip size 100k expire 30s store conn_rate(3s)

 

    # Allow clean known IPs to bypass the filter

    tcp-request connection accept if { src -f /etc/haproxy/whitelist.lst }

    # Shut the new connection as long as the client has already 10 opened

    tcp-request connection reject if { src_conn_rate ge 10 }

    tcp-request connection track-sc1 src

 

3.限制HTTP请求的的频率

frontend ft_web

    bind 0.0.0.0:8080

 

    # Use General Purpose Couter (gpc) 0 in SC1 as a global abuse counter

    # Monitors the number of request sent by an IP over a period of 10 seconds

    stick-table type ip size 1m expire 10s store gpc0,http_req_rate(10s)

    tcp-request connection track-sc1 src

    tcp-request connection reject if { src_get_gpc0 gt 0 }

 

backend bk_web

    balance roundrobin

    cookie MYSRV insert indirect nocache

   

    # If the source IP sent 10 or more http request over the defined period,

    # flag the IP as abuser on the frontend

    acl abuse src_http_req_rate(ft_web) ge 10

    acl flag_abuser src_inc_gpc0(ft_web)

    tcp-request content reject if abuse flag_abuser

  

    server srv1 192.168.1.2:80 check cookie srv1 maxconn 100

    server srv2 192.168.1.3:80 check cookie srv2 maxconn 100

 

4.haproxy的监控

 

hatop是一个用python语言编写的,交互式的ncurses客户端程序。

它的输出类似top程序,可以用来实时查看haproxy的状态,如果允许level admin则还可以enable,disable服务器。

 

[root@haproxy ~]# yum install socat

[root@haproxy ~]# wget http://hatop.googlecode.com/files/hatop-0.7.7.tar.gz

[root@haproxy ~]# tar xvzf hatop-0.7.7.tar.gz

[root@haproxy ~]# cd hatop-0.7.7

[root@haproxy ~]# install -m 755 bin/hatop /usr/local/bin

[root@haproxy ~]# install -m 644 man/hatop.1 /usr/local/share/man/man1

[root@haproxy ~]# gzip /usr/local/share/man/man1/hatop.1

 

[root@haproxy ~]# vi /etc/haproxy/haproxy.conf

在global段内加入如下:

stats socket /var/run/haproxy.stat mode 0600 level admin

 

重起haproxy

[root@haproxy ~]# /etc/init.d/haproxy reload

 

确认socket已建立

[root@haproxy ~]# ls -al /var/run/haproxy.stat

srw-------. 1 root root 0 Jan 15 20:53 haproxy.sock

 

运行hatop查看haproxy相关实时信息

[root@haproxy ~]# hatop -s /var/run/haproxy.stat

 

5.用Zabbix监控haproxy[http://www.juhonkoti.net/2010/10/15/script-and-template-to-export-data-from-haproxy-to-zabbix]

 

6.单网卡多个不同网段的相关配置

 [root@localhost examples]# vi /etc/iproute2/rt_tables

文件结尾追加如下内容:

64        CNC64

202      CNC202

211      CNC211

 

配置多路由表

[root@haproxy ~]# vi /etc/haproxy/haproxy.conf

#!/bin/bash

######

CNC64_IP="64.4.2.0/24"

CNC64_GW="64.4.2.1"

CNC202_IP="202.108.35.0/24"

CNC202_GW="202.108.1"

CNC211_IP="211.113.58.0/24"

CNC211_GW="211.113.58.1"

 

ip route flush table CNC64

ip route add default via $CNC64_GW dev eth0 table CNC64

ip rule add from $CNC64_IP table CNC64

 

ip route flush table CNC202

ip route add default via $CNC202_GW dev eth0 table CNC202

ip rule add from $CNC202_IP table CNC202

 

ip route flush table CNC211

ip route add default via $CNC211_GW dev eth0 table CNC211

ip rule add from $CNC211_IP table CNC211

 

修改keepalived配置文件

[root@haproxy ~]# vi /etc/haproxy/haproxy.conf

virtual_ipaddress_excluded {  # 超过20个VIP可以添加在virtual_ipaddress_excluded中,这些VIP不需要发送检测包

           64.4.2.111/24 dev eth0

           202.108.35.22/24 dev eth0

           211.113.58.7/24 dev eth0

        }

 

七、SSL offload配置（使用self-signed证书）

]# mkdir /etc/ssl

]# cd /etc/ssl

]# openssl genrsa -des3 -out server.key 1024

]# cp server.key server.key.orig

]# openssl rsa -in server.key.orig -out server.key  #去掉pravite key的passphrase

]# openssl req -new -key server.key -out server.csr

>Enter pass phrase for server.key:

>You are about to be asked to enter information that will be incorporated

>into your certificate request.

>What you are about to enter is what is called a Distinguished Name or a DN.

>There are quite a few fields but you can leave some blank

>For some fields there will be a default value,

>If you enter '.', the field will be left blank.

>-----

>Country Name (2 letter code) [XX]:US

>State or Province Name (full name) []:CA

>Locality Name (eg, city) [Default City]:Irvine

>Organization Name (eg, company) [Default Company Ltd]: Monster Inc.

>Organizational Unit Name (eg, section) []:

>Common Name (eg, your name or your server's hostname) []:*.monster.com

>Email Address []:

>

>Please enter the following 'extra' attributes

>to be sent with your certificate request

>A challenge password []:

>An optional company name []:

 

]# openssl x509 -req -days 365 -in server.csr \

                    -signkey server.key \

                    -out server.crt

 

]# cat server.crt server.key|tee server.pem

 

haproxy的相关配置:

frontend localhost

    bind *:80

    bind *:443 ssl crt /etc/ssl/server.pem

    redirect scheme https if !{ ssl_fc }

    mode http

    default_backend nodes

 

backend nodes

    mode http

    balance roundrobin

    option forwardfor

    option httpchk HEAD / HTTP/1.1\r\nHost:localhost

    server web01 172.17.0.3:9000 check

    server web02 172.17.0.3:9001 check

    server web03 172.17.0.3:9002 check

    http-request set-header X-Forwarded-Port %[dst_port]

    http-request add-header X-Forwarded-Proto https if { ssl_fc }

 

 2017.02.16 补充一个方便的技巧

haproxy官方提供了针对vim的语法文件，可以高亮显示keyword，对于修改配置文件来说很方便。

方法说一下：

1.将haproxy源码中example目录中的haproxy.vim复制到$HOME/.vim/syntax/

2.修改$HOME/.vimrc，加入:  au BufRead,BufNewFile haproxy* set ft=haproxy

 

八、系统安全加固

[root@haproxy ~]# yum install yum-remove-with-leaves

[root@haproxy ~]# yum remove gcc make

[root@haproxy ~]# vi remove-list

system-config-firewall-base

iptables-ipv6

dhcp-common

pciutils-libs

efibootmgr

dhclient

kernel-firmware

iwl5150-firmware

iwl6050-firmware

iwl6000g2a-firmware

iwl6000-firmware

ql2400-firmware

ql2100-firmware

libertas-usb8388-firmware

ql2500-firmware

zd1211-firmware

rt61pci-firmware

ql2200-firmware

ipw2100-firmware

ipw2200-firmware

iwl5000-firmware

ivtv-firmware

xorg-x11-drv-ati-firmware

atmel-firmware

iwl4965-firmware

iwl3945-firmware

rt73usb-firmware

ql23xx-firmware

bfa-firmware

iwl100-firmware

b43-openfwwf

aic94xx-firmware

iwl1000-firmware

[root@haproxy ~]# for I in `cat remove-list `;do yum -y remove $i;done

 

 

 

八、参考文档

1-http://mattiasgeniar.be/2010/11/04/a-custom-init-d-start-up-script-for-haproxy-start-stop-restart-reload-checkconfig/

2-http://www.snapt-ui.com/haproxy/simple-sysctl-tunings-for-haproxy/

3-https://gist.github.com/4039319

4-http://www.cyberciti.biz/files/linux-kernel/Documentation/networking/tproxy.txt

5-http://blog.exceliance.fr/2012/09/10/how-to-get-ssl-with-haproxy-getting-rid-of-stunnel-stud-nginx-or-pound/

6-http://www.symantec.com/connect/articles/apache-2-ssltls-step-step-part-2

7-http://www.igvita.com/2008/05/13/load-balancing-qos-with-haproxy/

8-http://h10025.www1.hp.com/ewfrf/wc/document?cc=us&lc=en&dlc=en&tmp_geoLoc=true&docname=c03561757

9-http://www.debuntu.org/how-to-log-haproxy-messages-only-once/#more-713

10-https://transloadit.com/blog/2010/08/haproxy-logging

11-http://kvz.io/blog/2010/08/11/haproxy-logging/

12-https://gist.github.com/1271962

13-http://www.rsyslog.com/doc/rsyslog_conf_actions.html

14-http://tehlose.wordpress.com/2011/10/10/a-log-file-for-each-virtual-host-with-haproxy-and-rsyslog/

15-http://jit.nuance9.com/2009/11/haproxy-routing-by-domain-name.html

16-http://unethicalblogger.com/2010/01/16/virtual-hosting-with-haproxy-and-wsgi.html

17-http://blog.silverbucket.net/post/31927044856/3-ways-to-configure-haproxy-for-websockets

18-http://blog.csdn.net/dog250/article/details/7107537

19-http://www.linuxjournal.com/content/monitoring-processes-kill

20-http://gurucollege.net/technology/ha-lamp-with-keepalived-pt2/

21-http://zauc.wordpress.com/2010/08/31/keepalived-conf之vrrp-instance部分解读/

22-http://interu.hatenablog.com/entry/20081024/1224784798

23-http://bbs.ywlm.net/thread-845-1-1.html

24-http://heylinux.com/archives/1942.html

25-http://www.intel.com/content/www/us/en/ethernet-controllers/82575-82576-82598-82599-ethernet-controllers-latency-appl-note.html

26-http://blog.csdn.net/turkeyzhou/article/details/7528182

27-http://www.vmware.com/files/pdf/techpaper/VMW-Tuning-Latency-Sensitive-Workloads.pdf

28-http://www.intel.com/support/cn/network/sb/cs-025829.htm

29-http://kaivanov.blogspot.kr/2015/02/keepalived-using-unicast-track-and.html

30-http://www.golinuxhub.com/2013/03/setting-up-custom-tcpip-keep-alive.html

31-https://serversforhackers.com/using-ssl-certificates-with-haproxy

32-https://www.digitalocean.com/community/tutorials/how-to-create-a-ssl-certificate-on-nginx-for-centos-6

33-http://man.lupaworld.com/content/manage/vi/doc/syntax.html