haproxy配置负载均衡（https）

haproxy配置负载均衡（https）

生成证书服务

安装服务

[root@DR req]# yum -y install openssl

创建根证书CA所需的目录及文件

[root@DR ~]# cd /etc/pki/
[root@DR pki]# ls
ca-trust  entitlement  product          rpm-gpg  tls
consumer  java         product-default  rsyslog
[root@DR pki]# mkdir CA
[root@DR pki]# cd CA/
[root@DR CA]# touch serial
[root@DR CA]# touch index.txt

指明证书的开始编号

[root@DR CA]# echo 01 > serial 

生成根证书的私钥（注意：私钥的文件名与存放位置要与配置文件中的设置相匹配）

[root@DR CA]# openssl genrsa -out /etc/pki/CA/private/cakey.pem 2048
Generating RSA private key, 2048 bit long modulus (2 primes)
............+++++
.......................................+++++
e is 65537 (0x010001)

生成自签证书，即根证书CA，自签证书的存放位置也要与配置文件中的设置匹配，生成证书时需 要填写相应的信息

[root@DR CA]# openssl req -new -x509 -key /etc/pki/CA/private/cakey.pem -out cacert.pem 
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:cn
State or Province Name (full name) []:hb
Locality Name (eg, city) [Default City]:wh
Organization Name (eg, company) [Default Company Ltd]:hs
Organizational Unit Name (eg, section) []:gzhy
Common Name (eg, your name or your server's hostname) []:localhost
Email Address []:

服务端（RS1与RS2）

创建证书存放路径并生成密钥

[root@RS1 httpd]# ls
conf  conf.d  conf.modules.d  logs  modules  run  state
[root@RS1 httpd]# mkdir ssl
[root@RS1 httpd]# cd ssl/
[root@RS1 ssl]# openssl genrsa -out test.key 2048
Generating RSA private key, 2048 bit long modulus (2 primes)
................................+++++
.............+++++
e is 65537 (0x010001)

百私钥文件和请求证书绑定在一起

[root@RS1 ssl]# openssl req -new -key test.key -out test.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:cn
State or Province Name (full name) []:hb
Locality Name (eg, city) [Default City]:hs
Organization Name (eg, company) [Default Company Ltd]:luolan
Organizational Unit Name (eg, section) []:gzhy
Common Name (eg, your name or your server's hostname) []:localhost
Email Address []:
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

[root@RS1 ssl]# ls
test.csr  test.key

把这个证书传到CA服务器根下面 ，去CA服务器上面创建一个req文件夹

[root@DR CA]# mkdir req

[root@RS1 ssl]# scp test.csr 192.168.240.50:/etc/pki/CA/req
The authenticity of host '192.168.240.50 (192.168.240.50)' can't be established.
ECDSA key fingerprint is SHA256:lJaDvQ9JrWLSAZbw6RRce5aPLnj7Mhatv99i5G9NbIw.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.240.50' (ECDSA) to the list of known hosts.
root@192.168.240.50's password: 
test.csr                               100%  989   430.7KB/s   00:00

apache服务端传过来的csr请求文件给CA服务器来颁 发

[root@DR req]# openssl ca -in /etc/pki/CA/req/test.csr -out /etc/pki/CA/req/test.crt  
Using configuration from /etc/pki/tls/openssl.cnf
Check that the request matches the signature
Signature ok
Certificate Details:
        Serial Number: 1 (0x1)
        Validity
            Not Before: Oct 17 02:32:02 2021 GMT
            Not After : Oct 17 02:32:02 2022 GMT
        Subject:
            countryName               = cn
            stateOrProvinceName       = hb
            organizationName          = hs
            organizationalUnitName    = gzhy
            commonName                = localhost
        X509v3 extensions:
            X509v3 Basic Constraints: 
                CA:FALSE
            Netscape Comment: 
                OpenSSL Generated Certificate
            X509v3 Subject Key Identifier: 
                B9:3E:C4:1B:5A:9B:01:2D:B7:3E:A6:62:25:97:0B:46:05:1A:F4:4A
            X509v3 Authority Key Identifier: 
                keyid:69:2E:C0:08:F6:38:1E:F1:84:03:C9:95:6A:38:66:72:E7:FE:2F:61

Certificate is to be certified until Oct 17 02:32:02 2022 GMT (365 days)
Sign the certificate? [y/n]:y


1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
[root@localhost req]# ls
test.crt  test.csr

把CA服务器上面的crt证书文件传送给apache服务器下面的/etc/httpd/ssl文件

[root@localhost req]# scp test.crt 192.168.240.40:/etc/httpd/ssl
The authenticity of host '192.168.240.40 (192.168.240.40)' can't be established.
ECDSA key fingerprint is SHA256:awIO+BpN0/pKDHYdOAaDx9Y3b7JiJMIP3xuYB2FQYoY.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.240.40' (ECDSA) to the list of known hosts.
root@192.168.240.40's password: 
test.crt                               100% 4396     3.4MB/s   00:00 

证书配置

安装软件

[root@RS1 ~]# yum -y install mod_ssl

查看密钥与证书文件

[root@RS1 ~]# ls /etc/httpd/ssl/
http.crt  http.csr  http.key

配置ssl

[root@RS1 ~]# vim /etc/httpd/conf.d/ssl.conf
·····
# General setup for the virtual host, inherited from global configuration
DocumentRoot "/var/www/html"    ##取消注释
ServerName www.example.com:443	##取消注释
·····

·····
SSLCertificateFile /etc/httpd/ssl/http.crt ##修改证书存放位置

#   Server Private Key:
#   If the key is not combined with the certificate, use this
#   directive to point at the key file.  Keep in mind that if
#   you've both a RSA and a DSA private key you can configure
#   both in parallel (to also allow the use of DSA ciphers, etc.)
#   ECC keys, when in use, can also be configured in parallel
SSLCertificateKeyFile /etc/httpd/ssl/http.key	##修改密钥存放位置
·······

重启httpd

[root@RS1 ~]# systemctl restart httpd
[root@RS1 ~]# ss -antl
State   Recv-Q  Send-Q   Local Address:Port   Peer Address:Port Process  
LISTEN  0       128            0.0.0.0:22          0.0.0.0:*             
LISTEN  0       128               [::]:22             [::]:*             
LISTEN  0       128                  *:443               *:*             
LISTEN  0       128                  *:80                *:*    

RS2重复安装以上步骤

追加文件测试

[root@RS1 html]# echo me 192.168.240.60 > index.html 
[root@RS1 html]# systemctl restart httpd

[root@RS2 html]#  echo me 192.168.240.40 > index.html
[root@RS2 html]# systemctl restart httpd

安装haproxy

三台主机提前安装http服务，并设置开机自启

[root@DR ~]# systemctl disable --now firewalld.service 
Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@DR ~]# setenforce 0
[root@DR ~]# systemctl enable --now httpd
Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service.

安装需要的工具

[root@DR ~]# yum -y install make gcc pcre-devel bzip2-devel openssl-devel systemd-devel
Repository extras is listed more than once in the configuration
上次元数据过期检查：0:07:16 前，执行于 2021年10月15日 星期五 02时54分03秒。
依赖关系解决。
=========================================================================
 软件包                 架构      版本                仓库          大小
=========================================================================
安装:
 bzip2-devel            x86_64    1.0.6-26.el8        baseos       224 k
······
  pcre2-utf32-10.32-2.el8.x86_64                                         
  systemd-devel-239-51.el8.x86_64                                        
  zlib-devel-1.2.11-17.el8.x86_64                                        

完毕！

提前准备安装压缩包并解压

[root@DR ~]# wget https://www.haproxy.org/download/2.4/src/haproxy-2.4.7.tar.gz
--2021-10-17 00:40:26--  https://www.haproxy.org/download/2.4/src/haproxy-2.4.7.tar.gz
正在解析主机 www.haproxy.org (www.haproxy.org)... 51.15.8.218, 2001:bc8:35ee:100::1
正在连接 www.haproxy.org (www.haproxy.org)|51.15.8.218|:443... 已连接。
已发出 HTTP 请求，正在等待回应... 200 OK
长度：3594183 (3.4M) [application/x-tar]
正在保存至: “haproxy-2.4.7.tar.gz.1”

haproxy-2.4.7.tar. 100%[=============>]   3.43M  1.33MB/s  用时 2.6s    

2021-10-17 00:40:35 (1.33 MB/s) - 已保存 “haproxy-2.4.7.tar.gz.1” [3594183/3594183])

创建haproxy用户

[root@DR ~]# useradd -r -M -s /sbin/nolong haproxy

进入解压目录进行编译安装

[root@DR ~]# cd /usr/local/haproxy/haproxy-2.4.0/
[root@DR haproxy]# make -j $(nproc) TARGET=linux-glibc \
> USE_OPENSSL=1 USE_PCRE=1 USE_SYSTEMD=1
[root@localhost haproxy-2.4.0]# make install PREFIX=/usr/local/haproxy
##安装完成
[root@localhost haproxy]# ls
doc  haproxy-2.4.0  haproxy2.4.0.tar.gz  sbin  share

设置内核参数

[root@DR ~]# echo net.ipv4.ip_nonlocal_bind =1 >> /etc/sysctl.conf 
[root@DR ~]# echo net.ipv4.ip_forward = 1 >> /etc/sysctl.conf 
[root@DR ~]# cat /etc/sysctl.conf 
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
net.ipv4.ip.nonlocal_bind =1
net.ipv4.ip_forward = 1 
[root@localhost ~]# sysctl -p
net.ipv4.ip_nonlocal_bind = 1
net.ipv4.ip_forward = 1

提供配置文件

[root@DR ~]# vim /etc/haproxy/haproxy.cfg
[root@DR ~]# cat /etc/haproxy/haproxy.cfg 
#--------------全局配置----------------
global
    log 127.0.0.1 local0  info
    #log loghost local0 info
    maxconn 20480
#chroot /usr/local/haproxy
    pidfile /var/run/haproxy.pid
    #maxconn 4000
    user haproxy
    group haproxy
    daemon
#---------------------------------------------------------------------
#common defaults that all the 'listen' and 'backend' sections will
#use if not designated in their block
#---------------------------------------------------------------------
defaults
    mode tcp
    log global
    option dontlognull
    option httpclose
    option httplog
    #option forwardfor
    option redispatch
    balance roundrobin
    timeout connect 10s
    timeout client 10s
    timeout server 10s
    timeout check 10s
    maxconn 60000
    retries 3
#--------------统计页面配置------------------
listen admin_stats
    bind 0.0.0.0:8189
    stats enable
    mode http
    log global
    stats uri /haproxy_stats
    stats realm Haproxy\ Statistics
    stats auth admin:admin
    #stats hide-version
    stats admin if TRUE
    stats refresh 30s
#---------------web设置-----------------------
listen webcluster
    bind 0.0.0.0:443
    mode tcp
    #option httpchk GET /index.html
    log global
    maxconn 3000
    balance roundrobin
    cookie SESSION_COOKIE insert indirect nocache
    server web01 192.168.240.40:443 check inter 2000 fall 5 
    server web02 192.168.240.60:443 check inter 2000 fall 5

haproxy.service文件编写

[root@DR ~]# vim /usr/lib/systemd/system/haproxy.service
[root@DR ~]# cat /usr/lib/systemd/system/haproxy.service 
[Unit]
Description=HAProxy Load Balancer
After=syslog.target network.target

[Service]
ExecStartPre=/usr/local/sbin/haproxy -f /etc/haproxy/haproxy.cfg   -c -q
ExecStart=/usr/local/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg  -p /var/run/haproxy.pid
ExecReload=/bin/kill -USR2 

[Install]
WantedBy=multi-user.target
[root@localhost ~]# systemctl daemon-reload

启用日志

[root@DR ~]# vim /etc/rsyslog.conf
·······
# Save boot messages also to boot.log
local0.*                   /var/log/haproxy.log
local7.*  
·····

重启服务

[root@DR ~]# systemctl restart rsyslog.service 
[root@DR ~]# systemctl restart haproxy.service 
[root@DR ~]# ss -antl
State               Recv-Q              Send-Q                           Local Address:Port                           Peer Address:Port             Process              
LISTEN              0                   128                                    0.0.0.0:22                                  0.0.0.0:*                                     
LISTEN              0                   128                                    0.0.0.0:443                                 0.0.0.0:*                                     
LISTEN              0                   128                                    0.0.0.0:8189                                0.0.0.0:*                                     
LISTEN              0                   128                                       [::]:22                                     [::]:*      

访问测试