随着互联网的快速发展,我们的web站点访问量和数据流量的快速增长,对于我们服务器的处理能里的要求也越来越高,这样的情况下,单一的服务器根本无法承受, 这样的话我们丢弃掉原有的设备,做硬件升级,会造成成本的浪费,如果再过一段时间,升级过后的硬件有负载不起了,怎么办呢?没关系,我们有负载均衡的技术,就不用担心了!
负载均衡通俗点说,就是一堆的计算机,或设备,同时为用户提供一个相同的服务,下面就来说说怎么实现的!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
首先在此感谢,"马哥"以及网友"IT你好" 的帮助,一路走来,问题不断,感谢朋友们的支持,再次表示谢谢, “刘庸有句话说的好,(人在黑暗既又邪恶的道路上成长,才能使自己变的更加强大)” 我想表达的意思就是,在学习的过程中遇到的问题越多,那么自己就积累的越多,当然对于那些勇于克服问题的童鞋而言,呵呵,闲话有点多了,下面来介绍下单一的nginx负载均衡是怎么工作的,
单一nginx负载均衡,见下图,
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1,首先用户发送请求访问bbs.andy.com,
2,当前端nginx负载均衡服务器(上图中的SVR1)收到用户的请求之后,nginx负载均衡器,会根据此前配置好的调度算法,代用户请求后端的应用程序服务器,
3,应用程序服务器(也就是上图中的SVR2,3,4) 收到前端nginx负载均衡器的请求的时候,它并不知道是nginx负载均衡器是代用户请求的,对于后端的应用程序服务器而言前端nginx负载均衡器就是一个用户, 那么它收到请求之后,将对应请求的处理结果,再返回给前端nginx负载均衡器,
4,当前端的nginx负载均衡器,收到后端应用程序服务器返回的响应内容之后再讲结果返回给用户,
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
应该是这样工作的吧,个人的理解,呵呵,说的不好,还望大家伙指点,下面来说说LNMP分布式架构是怎么工作的,也就是上图啦, 待会的配置也是根据上图的架构来配置的,
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
LNMP分布式架构的工作原理,根据上图来,
1,用户请求bbs.andy.com
2,前端nginx负载均衡器收到用户请求,根据此前设定的调度算法,代用户请求后端的应用程序服务器,假如说此时前端nginx负载均衡器,根据调度算法的结果,应该访问svr2,这台应用程序服务器,
3,那么svr2这台服务器根据用户请求的内容的不同,而进行不同的处理机制,根据上图的架构一般有两种可能,
当用户请求的内容是一个静态的html页面的话,那么svr2,就直接将请求的结果响应给前端nginx负载均衡器,
当用户请求的内容是一个动态的页面内容(在上图中也就是PHP页面了)那么svr2会去找svr4上面的FastCGI程序来帮忙解析动态php页面,如果此次页面请求中需要访问数据库的话,fastcgi会通过mysql的接口访问数据库的,然后将其请求的结果,返回给svr2,
当svr2收到返回的请求结果后,svr2再将结果返回给前端的nginx负载均衡器,
4,当前端的nginx负载均衡器收到请求的结果后,再将其返回给用户,
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
下面来配置下在nginx负载均衡的情况下,后端的LNMP应用服务器分布式架构,根据上图来,下面再简单介绍下上图中每台服务器的角色,已经IP地址,等信息,
服务器 角色 IP地址 DNS能解析的域名
SVR1 nginx负载均衡器 192.168.0.52/172.16.0.1 bbs.andy.com解析到192.168.0.52
SVR2 nginx 172.16.0.2
SVR3 nginx/NFS 172.16.0.3
SVR4 php(FastCGI)/MySQL 172.16.0.4
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SVR1负责接受用户请求,根据调度算法,负载到SVR2 SVR3上面去,
SVR3上面有个NFS服务,是提供共享存储的,这里主要是共享网站的源码
SVR4 php(FastCGI)提供SVR2 SVR3的动态PHP页面的请求,
SVR4 MYSQL数据库提供SVR4上的PHP(FastCGI)的数据请求等,
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
单一nginx负载均衡+LNMP分布式架构 rhel5.8的系统上实现
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
目录
一,安装配置SVR4
二,安装配置SVR3
三,安装配置SVR2
四,安装配置SVR1注:配置过程中需要用到的源码包,自己在官方下载, 所有包下载至系统的/usr/src目录
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
一,安装配置SVR4
1.安装基本开发库,等依赖软件包
#yum -y groupinstall "Development Libraries" "Development Tools" "X Software Development"
2.安装配置mysql
2.1 编译安装mysql
#cd /usr/src
#tar xzvf mysql-5.0.56.tar.gz
#cd mysql-5.0.56
#./configure --prefix=/usr/local/mysql--sysconfdir=/usr/local/mysql/etc --with-ssl \
--localstatedir=/usr/local/mysql/database --enable-assembler --with-readline \
--with-extra-charsets=complex--enable-thread-safe-client --with-big-tables \
--with-embedded-server --enable-local-infile --with-plugins=innobase
#make && make install
2.2 将mysql命令加入到系统搜索路径
#vim /etc/profile
PATH=/usr/local/mysql/bin:$PATH 将次行添加到文件中的44行
#export PATH=/usr/local/mysql/bin:$PATH
2.3 将头文件添加到系统搜索路径
#ln -s /usr/local/mysql/include/mysql /usr/include/mysql
2.4 将库文件添加到系统搜索路径
#echo “/usr/local/mysql/lib/mysql”>/etc/ld.so.conf.d/mysql.conf
#ldconfig
2.5 为mysql提供配置文件,服务启动脚本,用户,并初始化mysql
#cp support-files/my-huge.cnf /etc/my.cnf
#cp support-files/mysql.server /etc/init.d/mysqld
#chmod a+x /etc/init.d/mysqld
#useradd -s /sbin/nologin mysql
#chown -R mysql:mysql /usr/local/mysql
#mysql_install_db --user=mysql
#chkconfig --add mysqld
#chkconfig mysqld on
#service mysqld restart
3,安装配置php(FastCGI)
3.1 编译安装libiconv 编码转换库
#cd /usr/src
#tar xzvf libiconv-1.13.1.tar.gz
#cd libiconv-1.13.1
#./configure --prefix=/usr/local
#make && make install
3.2 编译安装libmcryp加密算法扩展库
#cd /usr/src
#tar xjvf libmcrypt-2.5.8.tar.bz2
#cd libmcrypt-2.5.8
#./configure
#make
#make install
#ldconfig
#cd libltdl/
#./configure --enable-ltdl-install
#make &&make install
3.3 编译安装mhash加密算法扩展库
#cd /usr/src
#tar xjvf mhash-0.9.9.9.tar.bz2
#cd mhash-0.9.9.9
#./configure
#make &&make install
#ln -s /usr/local/lib/libmcrypt.la /usr/lib/libmcrypt.la
#ln -s /usr/local/lib/libmcrypt.so /usr/lib/libmcrypt.so
#ln -s /usr/local/lib/libmcrypt.so.4 /usr/lib/libmcrypt.so.4
#ln -s /usr/local/lib/libmcrypt.so.4.4.8 /usr/lib/libmcrypt.so.4.4.8
#ln -s /usr/local/lib/libmhash.a /usr/lib/libmhash.a
#ln -s /usr/local/lib/libmhash.la /usr/lib/libmhash.la
#ln -s /usr/local/lib/libmhash.so /usr/lib/libmhash.so
#ln -s /usr/local/lib/libmhash.so.2 /usr/lib/libmhash.so.2
#ln -s /usr/local/lib/libmhash.so.2.0.1 /usr/lib/libmhash.so.2.0.1
3.4 编译安装mcrypt 加密算法工具
#cd /usr/src
#tar xzvf mcrypt-2.6.8.tar.gz
#cd mcrypt-2.6.8
#./configure
#make && make install
3.5 编译安装php(FastCGI) 这里使用的是5.4的版本, 5.4的版本不需要打fpm补丁就支持FastCGI
3.5.1 编译安装php
#cd /usr/src
#tar xjvf php-5.4.4.tar.bz2
#cd php-5.4.4
#./configure --prefix=/usr/local/php--with-mysql=/usr/local/mysql/ --with-openssl \
--enable-fpm --enable-sockets --enable-sysvshm--with-mysqli=/usr/local/mysql/bin/mysql_config\
--enable-mbstring --with-freetype-dir --with-jpeg-dir --with-png-dir --with-zlib-dir \
--with-libxml-dir=/usr/ --enable-xml --with-mhash --with-mcrypt--with-config-file-path=/etc \
--with-config-file-scan-dir=/etc/php.d --with-bz2 --with-curl --with-ldap --with-iconv-dir
#make ZEND_EXTRA_LIBS='-liconv'
#make install
3.5.2 为php-fpm提供配置文件,服务启动脚本等,
#cp php.ini-production /etc/php.ini
#cp sapi/fpm/init.d.php-fpm /etc/rc.d/init.d/php-fpm
#chmod +x /etc/rc.d/init.d/php-fpm
#chkconfig --add php-fpm
#chkconfig php-fpm on
#cp /usr/local/php/etc/php-fpm.conf.default /usr/local/php/etc/php-fpm.conf
#vim /usr/local/php/etc/php-fpm.conf 修改内容如下
pid= /usr/local/php/var/run/php-fpm.pid
listen = 172.16.0.4:9000
pm.max_children=50
pm.start_servers=5
pm.min_spare_servers=2
pm.max_spare_servers=8
#service php-fpm start
二,安装配置SVR3
1.安装基本开发库,及依赖软件包
#yum groupinstall "Development Libraries" "Development Tools"
#yum -y install pcre-devel
2.安装配置nginx
2.1 编译安装nginx
#useradd -s /sbin/nologin nginx
#cd /usr/src
#tar xzvf nginx-1.2.2.tar.gz
#cd nginx-1.2.2
#./configure --prefix=/usr/local/nginx--pid-path=/var/run/nginx/nginx.pid \
--lock-path=/var/lock/nginx.lock--user=nginx--group=nginx--with-http_ssl_module \
--with-http_flv_module --with-http_stub_status_module --with-http_gzip_static_module \
--http-client-body-temp-path=/var/tmp/nginx/client/--http-proxy-temp-path=/var/tmp/nginx/proxy \
--http-fastcgi-temp-path=/var/tmp/nginx/fcgi--http-uwsgi-temp-path=/var/tmp/nginx/uwsgi \
--http-scgi-temp-path=/var/tmp/nginx/scgi --with-pcre
#make &&make install
2.2 为nginx提供服务启动脚本
#vim /etc/rc.d/init.d/nginx 内容如下
#!/bin/sh
#
# nginx - this script starts and stops the nginx daemon
#
# chkconfig: - 85 15
# description: Nginx is an HTTP(S) server, HTTP(S) reverse \
# proxy and IMAP/POP3 proxy server
# processname: nginx
# config: /etc/nginx/nginx.conf
# config: /etc/sysconfig/nginx
# pidfile: /var/run/nginx.pid
# Source function library.
. /etc/rc.d/init.d/functions
# Source networking configuration.
. /etc/sysconfig/network
# Check that networking is up.
[ "$NETWORKING" = "no" ] && exit 0
nginx="/usr/local/nginx/sbin/nginx"
prog=$(basename $nginx)
NGINX_CONF_FILE="/usr/local/nginx/conf/nginx.conf"
[ -f /etc/sysconfig/nginx ] && . /etc/sysconfig/nginx
lockfile=/var/lock/subsys/nginx
make_dirs() {
# make required directories
user=`nginx -V 2>&1 | grep "configure arguments:" | sed 's/[^*]*--user=\([^ ]*\).*/\1/g' -`
options=`$nginx -V 2>&1 | grep 'configure arguments:'`
for opt in $options; do
if [ `echo $opt | grep '.*-temp-path'` ]; then
value=`echo $opt | cut -d "=" -f 2`
if [ ! -d "$value" ]; then
# echo "creating" $value
mkdir -p $value && chown -R $user $value
fi
fi
done
}
start() {
[ -x $nginx ] || exit 5
[ -f $NGINX_CONF_FILE ] || exit 6
make_dirs
echo -n $"Starting $prog: "
daemon $nginx -c $NGINX_CONF_FILE
retval=$?
echo
[ $retval -eq 0 ] && touch $lockfile
return $retval
}
stop() {
echo -n $"Stopping $prog: "
killproc $prog -QUIT
retval=$?
echo
[ $retval -eq 0 ] && rm -f $lockfile
return $retval
}
restart() {
configtest || return $?
stop
sleep 1
start
}
reload() {
configtest || return $?
echo -n $"Reloading $prog: "
killproc $nginx -HUP
RETVAL=$?
echo
}
force_reload() {
restart
}
configtest() {
$nginx -t -c $NGINX_CONF_FILE
}
rh_status() {
status $prog
}
rh_status_q() {
rh_status >/dev/null 2>&1
}
case "$1" in
start)
rh_status_q && exit 0
$1
;;
stop)
rh_status_q || exit 0
$1
;;
restart|configtest)
$1
;;
reload)
rh_status_q || exit 7
$1
;;
force-reload)
force_reload
;;
status)
rh_status
;;
condrestart|try-restart)
rh_status_q || exit 0
;;
*)
echo $"Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest}"
exit 2
esac
#chmod a+x /etc/init.d/nginx
#chkconfig --add nginx
#chkconfig nginx on
2.3 编辑nginx的主配置文件
#vim /usr/local/nginx/conf/nginx.conf 内容如下
user nginx;
worker_processes 10;
error_log logs/error.log crit;
pid logs/nginx.pid;
events
{
use epoll;
worker_connections 51000;
}
http {
include mime.types;
default_type application/octet-stream;
client_header_buffer_size 32k;
large_client_header_buffers 4 32k;
client_max_body_size 10m;
sendfile on;
tcp_nopush on;
keepalive_timeout 60;
tcp_nodelay on;
fastcgi_connect_timeout 300;
fastcgi_send_timeout 300;
fastcgi_read_timeout 300;
fastcgi_buffer_size 64k;
fastcgi_buffers 4 64k;
fastcgi_busy_buffers_size 128k;
fastcgi_temp_file_write_size 128k;
gzip on;
gzip_min_length 1k;
gzip_buffers 4 16k;
gzip_http_version 1.0;
gzip_comp_level 2;
gzip_types text/plain application/x-javascript text/ccs application/xml;
gzip_vary on;
server {
listen 80;
server_name 172.16.0.3;
index index.html index.htm index.php;
root /web/bbs;
location ~ .*\.(php|php5)?$
{
fastcgi_pass 172.16.0.4:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /web/bbs$fastcgi_script_name;
include fastcgi.conf;
}
access_log logs/bbs.log;
}
}
2.4 创建网站根目录以及测试文件
#mkdir -pv /web/bbs
#vim /web/bbs/index.php
test
phpinfo();
?>
2.5 启动nginx服务
#service nginx restart
3,配置NFS
#vim /etc/exports
/web 172.16.0.*(rw,no_root_squash,sync)
#chkconfig portmap on
#chkconfig nfs on
#service portmap restart
#service nfs restart
三,安装配置SVR2
1.安装基本开发库,及依赖软件包
#yum groupinstall "Development Libraries" "Development Tools"
#yum -y install pcre-devel
2.安装配置nginx
2.1 编译安装nginx
#useradd -s /sbin/nologin nginx
#cd /usr/src
#tar xzvf nginx-1.2.2.tar.gz
#cd nginx-1.2.2
#./configure --prefix=/usr/local/nginx --pid-path=/var/run/nginx/nginx.pid \
--lock-path=/var/lock/nginx.lock --user=nginx --group=nginx --with-http_ssl_module \
--with-http_flv_module --with-http_stub_status_module --with-http_gzip_static_module \
--http-client-body-temp-path=/var/tmp/nginx/client/ --http-proxy-temp-path=/var/tmp/nginx/proxy \
--http-fastcgi-temp-path=/var/tmp/nginx/fcgi --http-uwsgi-temp-path=/var/tmp/nginx/uwsgi \
--http-scgi-temp-path=/var/tmp/nginx/scgi --with-pcre
#make && make install
2.2 为nginx提供服务启动脚本
见 二.2.2 的配置内容
2.3 编辑nginx的主配置文件
#vim /usr/local/nginx/conf/nginx.conf 内容如下
user nginx;
worker_processes 10;
error_log logs/error.log crit;
pid logs/nginx.pid;
events
{
use epoll;
worker_connections 51000;
}
http {
include mime.types;
default_type application/octet-stream;
client_header_buffer_size 32k;
large_client_header_buffers 4 32k;
client_max_body_size 10m;
sendfile on;
tcp_nopush on;
keepalive_timeout 60;
tcp_nodelay on;
fastcgi_connect_timeout 300;
fastcgi_send_timeout 300;
fastcgi_read_timeout 300;
fastcgi_buffer_size 64k;
fastcgi_buffers 4 64k;
fastcgi_busy_buffers_size 128k;
fastcgi_temp_file_write_size 128k;
gzip on;
gzip_min_length 1k;
gzip_buffers 4 16k;
gzip_http_version 1.0;
gzip_comp_level 2;
gzip_types text/plain application/x-javascript text/ccs application/xml;
gzip_vary on;
server {
listen 80;
server_name 172.16.0.2;
index index.html index.htm index.php;
root /web/bbs;
location ~ .*\.(php|php5)?$
{
fastcgi_pass 172.16.0.4:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /web/bbs$fastcgi_script_name;
include fastcgi.conf;
}
access_log logs/bbs.log;
}
}
3.重启nginx服务
#service nginx restart
4.使用NFS 共享的网站目录,以达到共享存储的目的
#mkdir /web
#mount -t nfs 172.16.0.3:/web /web
#echo "mount -t nfs 172.16.0.3:/web /web " >> /etc/rc.local
四.安装配置SVR1
1. 安装基本开发库,以及依赖的软件包
#yum groupinstall "Development Libraries" "Development Tools"
#yum -y install pcre-devel
2.安装配置nginx
useradd -s /sbin/nologin nginx
#cd /usr/src
#tar xzvf nginx-1.2.2.tar.gz
#cd nginx-1.2.2
#./configure --prefix=/usr/local/nginx--pid-path=/var/run/nginx/nginx.pid \
--lock-path=/var/lock/nginx.lock--user=nginx--group=nginx--with-http_ssl_module \
--with-http_flv_module --with-http_stub_status_module --with-http_gzip_static_module \
--http-client-body-temp-path=/var/tmp/nginx/client/--http-proxy-temp-path=/var/tmp/nginx/proxy \
--http-fastcgi-temp-path=/var/tmp/nginx/fcgi--http-uwsgi-temp-path=/var/tmp/nginx/uwsgi \
--http-scgi-temp-path=/var/tmp/nginx/scgi --with-pcre
#make && make install
3.为nginx提供服务启动脚本
见 二.2.2 的配置内容
4.编辑nginx主配置文件,配置负载均衡
#vim /usr/local/nginx/conf/nginx.conf
user nginx;
worker_processes 10;
error_log logs/error.log crit;
pid logs/nginx.pid;
events
{
use epoll;
worker_connections 51000;
}
http {
include mime.types;
default_type application/octet-stream;
keepalive_timeout 60;
tcp_nodelay on;
#指定负载均衡的方式
upstream bbs.andy.com {
server 172.16.0.2:80;
server 172.16.0.3:80;
ip_hash;
}
server {
listen 80;
server_name bbs.andy.com;
index index.html index.htm index.php;
location / {
proxy_pass http://bbs.andy.com;
proxy_next_upstream http_502 http_504 error timeout invalid_header;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 600;
proxy_read_timeout 600;
proxy_send_timeout 600;
proxy_buffer_size 8k;
proxy_temp_file_write_size 64k;
}
access_log logs/bbs.log;
}
}
3.重启nginx服务
#service nginx restart
五,收尾配置,
1,需要在每台服务器上执行以下命令
#chkconfig iptables off
#service iptables stop
#setenforce 0
#echo "setenforce 0" >>/etc/rc.local
2.需要在SVR4上挂载nfs共享
#mkdir /web
#mount -t nfs 172.16.0.3:/web /web
#echo "mount -t nfs 172.16.0.3:/web /web" >>/etc/rc.local
OK.到这里配置就结束了,
2012年7月16日凌晨3:00整 完!