1 概述

当代理服务器数量增加时，单台puppetmaster压力增大，会出现解析缓慢，甚至出现“time out”之类报错的错误。Nginx工作于网络的七层之上，可以作为负载均衡服务器。利用Load Balancing可以优化上述出现的问题。


2 服务器环境及软件版本

server version：CentOS6.5 i386

Ruby：ruby-1.8.7.352-13.el6.i686

Puppet: puppet-2.7.23

Nginx: nginx-1.4.4


3 安装Mongrel

通过指定mongrel类型来使用puppet多端口配置

yum install -y rubygem-mongrel


4 配置puppetmaster

vim /etc/sysconfig/puppetmaster，添加以下两行,分别代表多端口、mongrel类型

PUPPETMASTER_PORTS=(8141 8142 8143 8144 8145 )

PUPPETMASTER_EXTRA_OPTS="--servertype=mongrel --ssl_client_header=HTTP_X_SSL_SUBJECT"


5 安装Nginx服务

安装之前请确保系统已经安装pcre-devel正则库，然后再编译安装Nginx，需要添加SSL模块参数支持，Nginx的安装过程如下所示：

yum -y install pcre-devel

cd $hd

wget http://nginx.org/download/nginx-1.4.4.tar.gz

tar zxvf nginx-1.4.4.tar.gz

cd nginx-1.4.4

./configure --prefix=/usr/local/nginx --with-http_ssl_module

make && make install

添加www用户组及用户：

groupadd www

useradd -g www www


6 配置Nginx

vim /usr/local/nginx/conf/nginx.conf：


user www;

worker_processes 8;

events{

   worker_connections 65535;

}

http{

   include      mime.types;

   default_type application/octet-stream;

#定义puppet客户端访问puppet-server端日志格式  

log_format main  '$remote_addr - $remote_user [$time_local] "$request"$request_length $request_time $time_local'

   '$status$body_bytes_sent $bytes_sent $connection $msec "$http_referer"'      '"$http_user_agent""$http_x_forwarded_for" upstream_response_time$upstream_addr $upstream_status ';


   access_log /usr/local/nginx/logs/access.log  main;


   sendfile       on;

   tcp_nopush    on;

   keepalive_timeout 65;

   upstreampuppetmaster {

       server127.0.0.1:8141;

       server127.0.0.1:8142;

       server127.0.0.1:8143;

       server127.0.0.1:8144;

       server127.0.0.1:8145;

   }

   server{

       listen      8140;

       root /etc/puppet;

       ssl on;

       ssl_session_timeout 5m;

   #如下为puppemaster服务器端证书地址

   ssl_certificate /var/lib/puppet/ssl/certs/rango.fugue.com.pem;

   ssl_certificate_key /var/lib/puppet/ssl/private_keys/rango.fugue.com.pem;

   ssl_client_certificate /var/lib/puppet/ssl/ca/ca_crt.pem;

   ssl_crl /var/lib/puppet/ssl/ca/ca_crl.pem;

   ssl_verify_client optional;

   #File sections

   location /production/file_content/files/ {

   types{ }

   default_type application/x-raw;

   #定义puppet推送路径别名

   alias /etc/puppet/files/;

   }


   #Modules files sections

   location ~ /production/file_content/modules/.+/ {

   root /etc/puppet/modules;

   types{ }

   default_type application/x-raw;

   rewrite ^/production/file_content/modules/(.+)/(.+)$ /$1/files/$2 break; }

location/ {

   ##设置跳转到puppetmaster负载均衡

   proxy_pass http://puppetmaster;

   proxy_redirect off;

   proxy_set_header Host $host;

   proxy_set_header X-Real-IP $remote_addr;

   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

   proxy_set_header X-Client-Verify $ssl_client_verify;

   proxy_set_header X-SSL-Subject $ssl_client_s_dn;

   proxy_set_header X-SSL-Issuer $ssl_client_i_dn;

   proxy_buffer_size 10m;

   proxy_buffers 1024 10m;

   proxy_busy_buffers_size 10m;

   proxy_temp_file_write_size 10m;

   proxy_read_timeout 120;

   }

   }

}


7 启动Nginx及puppet-server

7.1 首先关闭puppetmaster进程：/etc/init.d/puppetmaster stop


7.2 启动Nginx

/usr/local/nginx/sbin/nginx

nginx占用puppetmaster默认的8140端口后，用如下命令来检查8140端口是否被nginx接管：

lsof-i:8140

此命令显示结果表明8140被nginx进程接管：

COMMAND  PID USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME

nginx  12433 root    6u  IPv4 1309402      0t0  TCP *:8140 (LISTEN)

nginx  12434  www    6u  IPv4 1309402      0t0  TCP *:8140 (LISTEN)

nginx  12435  www    6u  IPv4 1309402      0t0  TCP *:8140 (LISTEN)

nginx  12436  www    6u  IPv4 1309402      0t0  TCP *:8140 (LISTEN)

nginx  12437  www    6u  IPv4 1309402      0t0  TCP *:8140 (LISTEN)

nginx  12438  www    6u  IPv4 1309402      0t0  TCP *:8140 (LISTEN)

nginx  12439  www    6u  IPv4 1309402      0t0  TCP *:8140 (LISTEN)

nginx  12440  www    6u  IPv4 1309402      0t0  TCP *:8140 (LISTEN)

nginx  12441  www    6u  IPv4 1309402      0t0  TCP *:8140 (LISTEN)


7.3 启动puppetmaster

/etc/init.d/puppetmaster start

Starting puppetmaster:

Port:8141                                                 [  OK  ]

Port:8142                                                 [  OK  ]

Port:8143                                                 [  OK  ]

Port:8144                                                 [  OK  ]

Port:8145                                                 [  OK  ]


8 总结

本文旨在讲述通过基于网络七层的软件负载均衡技术Nginx，结合puppet的多端口设置来构建puppet的load balancing网络，达到优化puppet吞吐，缓解解析压力的目的。后续文章将着力于puppet report相关的内容。

                                                                                                                      ——RangoChen