nginx反向代理与负载均衡
第1章 软硬件准备
1.1 硬件准备
1.1.1 准备4台VM虚拟机,两台做负载均衡,两台做web
hostname ip 说明
lb01 10.0.0.80 Nginx主负载均衡
lb02 10.0.0.81 Nginx辅负载均衡
Web01 10.0.0.82 Web01服务器
Web02 10.0.0.83 Web02服务器
1.2 软件准备
系统:CentOS6.9 x86_64
软件:wget http://nginx.org/download/nginx-1.6.3.tar.gz
第2章 安装nginx软件
2.1.1 安装nginx所需的pcre库
[root@localhost ~]# yum -y install pcre pcre-devel
[root@localhost ~]# rpm -qa pcre pcre-devel
pcre-7.8-7.el6.x86_64
pcre-devel-7.8-7.el6.x86_64
2.1.2 安装openssl-devel
[root@localhost ~]# yum -y install openssl openssl-devel
[root@localhost ~]# rpm -qa openssl openssl-devel
openssl-devel-1.0.1e-57.el6.x86_64
openssl-1.0.1e-57.el6.x86_64
2.2 安装nginx
2.2.1 开始安装nginx
mkdir -p /home/tools 创建一个放软件的目录
wget http://nginx.org/download/nginx-1.6.3.tar.gz 软件下载地址
[root@localhost tools]# ls -l
total 788
-rw-r--r-- 1 root root 805253 Apr 8 2015 nginx-1.6.3.tar.gz
[root@localhost tools]# useradd nginx -s /sbin/nologin –M 创建一个用户
[root@localhost tools]# tar -xf nginx-1.6.3.tar.gz 解压文件
[root@localhost tools]# ls
nginx-1.6.3 nginx-1.6.3.tar.gz
[root@localhost tools]# cd nginx-1.6.3 进入解压后的软件目录
2.2.2 编译参数
[root@localhost nginx-1.6.3]# ./configure --user=nginx --group=nginx --prefix=/application/nginx-1.6.3/ --with-http_stub_status_module --with-http_ssl_module
--user=nginx 进程用户权限
--group=nginx 进程用户组权限
--prefix=/application/nginx-1.6.3/ nginx安装位置
--with-http_stub_status_module 激活状态信息
--with-http_ssl_module 激活ssl功能
2.2.3 make
2.2.3.1 出现以下信息说明正确
sed -e "s|%%PREFIX%%|/application/nginx-1.6.3/|" \
-e "s|%%PID_PATH%%|/application/nginx-1.6.3//logs/nginx.pid|" \
-e "s|%%CONF_PATH%%|/application/nginx-1.6.3//conf/nginx.conf|" \
-e "s|%%ERROR_LOG_PATH%%|/application/nginx-1.6.3//logs/error.log|" \
< man/nginx.8 > objs/nginx.8
make[1]: Leaving directory `/home/tools/nginx-1.6.3'
2.2.4 make install
2.2.4.1 出现以下信息说明正确
cp conf/nginx.conf '/application/nginx-1.6.3//conf/nginx.conf.default'
test -d '/application/nginx-1.6.3//logs' || mkdir -p '/application/nginx-1.6.3//logs'
test -d '/application/nginx-1.6.3//logs' || mkdir -p '/application/nginx-1.6.3//logs'
test -d '/application/nginx-1.6.3//html' || cp -R html '/application/nginx-1.6.3/'
test -d '/application/nginx-1.6.3//logs' || mkdir -p '/application/nginx-1.6.3//logs'
make[1]: Leaving directory `/home/tools/nginx-1.6.3'
2.2.4.2 创建一条软连接,方便使用
[root@localhost nginx-1.6.3]# ln -s /application/nginx-1.6.3/ /application/nginx
安装时指定版本号为了便于查看当前使用的nginx版本,也方便升级
内部人员使用路径/application/nginx
当nginx软件升级编译成新版本号的版本后,删除原来的软连接,再重新建立新的连接到/application/nginx的软连接即可
程序中如果有引用nginx的路径,不需要做任何更改,因为升级后访问的路径还是/application/nginx
2.2.4.3 检查连接及目录状态
[root@localhost nginx-1.6.3]# ll /application/|grep nginx
lrwxrwxrwx 1 root root 25 Aug 3 06:11 nginx -> /application/nginx-1.6.3/
drwxr-xr-x 6 root root 4096 Aug 3 06:09 nginx-1.6.3
[root@localhost nginx-1.6.3]# ls -l /application/nginx
lrwxrwxrwx 1 root root 25 Aug 3 06:11 /application/nginx -> /application/nginx-1.6.3/
2.2.4.4 nginx目录中的内容
[root@localhost nginx-1.6.3]# ls -l /application/nginx/
total 16
drwxr-xr-x 2 root root 4096 Aug 3 06:09 conf
drwxr-xr-x 2 root root 4096 Aug 3 06:09 html
drwxr-xr-x 2 root root 4096 Aug 3 06:09 logs
drwxr-xr-x 2 root root 4096 Aug 3 06:09 sbin
2.3 启动并检查安装结果
2.3.1 启动前检查配置文件的语法
[root@localhost nginx-1.6.3]# /application/nginx/sbin/nginx -t
nginx: the configuration file /application/nginx-1.6.3//conf/nginx.conf syntax is ok
nginx: configuration file /application/nginx-1.6.3//conf/nginx.conf test is successful
2.3.2 启动nginx服务
[root@localhost nginx-1.6.3]# /application/nginx/sbin/nginx
2.3.3 查看nginx服务对应的端口是否成功启动
[root@localhost nginx-1.6.3]# ss -tlunp|grep nginx
tcp LISTEN 0 128 *:80 *:* users:(("nginx",30949,6),("nginx",30950,6))
2.3.4 检查nginx的实际效果
2.3.4.1 打开windows的浏览器,出现以下信息就ok
nginx安装完毕
2.4 配置用于测试的web服务
2.4.1 nginx web01和web02的配置如下
[root@localhost conf]# cat nginx.conf
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
sendfile on;
keepalive_timeout 65;
server {
listen 80;
server_name bbs.tiandi.com;
location / {
root html/bbs;
index index.html index.htm;
}
access_log logs/access_bbs.log main;
}
server {
listen 80;
server_name www.tiandi.com;
location / {
root html/www;
index index.html index.htm;
}
access_log logs/access_www.log main;
}
}
2.4.2 配置完成后检查语法,并启动nginx服务(web01和web02操作相同)
[root@localhost conf]# /application/nginx/sbin/nginx –t
nginx: the configuration file /application/nginx-1.6.3//conf/nginx.conf syntax is ok
nginx: configuration file /application/nginx-1.6.3//conf/nginx.conf test is successful
[root@localhost conf]# /application/nginx/sbin/nginx -s reload
[root@localhost conf]# ss -tlunp |grep nginx
tcp LISTEN 0 128 *:80 *:* users:(("nginx",5169,6),("nginx",5319,6))
2.4.3 填充测试文件数据(web01上操作)
[root@localhost conf]# mkdir -p /application/nginx/html/{www,bbs}
[root@localhost html]# echo '10.0.0.82 www' >www/index.html
[root@localhost html]# echo '10.0.0.82 bbs' >bbs/index.html
[root@localhost html]# cat www/index.html
10.0.0.82 www
[root@localhost html]# cat bbs/index.html
10.0.0.82 bbs
2.4.4 填充测试文件数据(web02上操作)
[root@localhost conf]# mkdir -p /application/nginx/html/{www,bbs}
[root@localhost html]# echo '10.0.0.83 www' >www/index.html
[root@localhost html]# echo '10.0.0.83 bbs' >bbs/index.html
[root@localhost html]# cat www/index.html
10.0.0.83 www
[root@localhost html]# cat bbs/index.html
10.0.0.83 bbs
2.4.4.1 配置解析web01的ip和主机名后,用curl简单测试web01
[root@lb01 ~]# tail -2 /etc/hosts
10.0.0.82 www.tiandi.com
10.0.0.82 bbs.tiandi.com
[root@lb01 ~]# curl www.tiandi.com
10.0.0.82 www
[root@lb01 ~]# curl bbs.tiandi.com
10.0.0.82 bbs
2.4.4.2 配置解析web01的ip和主机名后,用curl简单测试web02
[root@lb01 ~]# tail -2 /etc/hosts
10.0.0.83 www.tiandi.com
10.0.0.83 bbs.tiandi.com
[root@lb01 ~]# curl www.tiandi.com
10.0.0.83 www
[root@lb01 ~]# curl bbs.tiandi.com
10.0.0.83 bbs
2.5 实现一个简单的负载均衡
hostname ip 说明
lb01 10.0.0.80 Nginx主负载均衡
下面进行一个简单的nginx负载均衡配置,代理www.tiandi.com服务,节点为web01和web02
2.5.1 nginx.conf配置文件内容如下:
[root@lb01 conf]# cat nginx.conf
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
upstream www_server_pools { 定义web服务器池,包含了82,83两个web节点
server 10.0.0.82:80 weight=1;
server 10.0.0.83:80 weight=1;
}
server {
listen 80;
server_name www.tiandi.com;
location / {
proxy_pass http://www_server_pools; 访问www.tiandi.com,请求发给www_server_pools里面的节点
}
}
}
2.5.2 检查语法并重新加载nginx
[root@lb01 conf]# ../sbin/nginx -t
nginx: the configuration file /application/nginx-1.6.3//conf/nginx.conf syntax is ok
nginx: configuration file /application/nginx-1.6.3//conf/nginx.conf test is successful
[root@lb01 conf]# ../sbin/nginx -s reload
2.5.2.1 检查负载均衡测试结果,linux作为客户端的测试结果如下
[root@lb01 conf]# tail -1 /etc/hosts
10.0.0.80 www.tiandi.com
[root@lb01 conf]# curl www.tiandi.com
10.0.0.82 bbs
[root@lb01 conf]# curl www.tiandi.com
10.0.0.83 bbs
[root@lb01 conf]# curl www.tiandi.com
10.0.0.82 bbs
[root@lb01 conf]# curl www.tiandi.com
10.0.0.83 bbs
[root@lb01 conf]# curl www.tiandi.com
10.0.0.82 bbs
[root@lb01 conf]# curl www.tiandi.com
10.0.0.83 bbs
2.5.3 宕掉任意一台web节点,查看测试情况
[root@lb01 conf]# curl www.tiandi.com
10.0.0.83 bbs
[root@lb01 conf]# curl www.tiandi.com
10.0.0.83 bbs
[root@lb01 conf]# curl www.tiandi.com
10.0.0.83 bbs
[root@lb01 conf]# curl www.tiandi.com
10.0.0.83 bbs
# 当宕掉一台的时候业务不受影响
2.5.4 宕掉所有web节点
[root@lb01 conf]# curl www.tiandi.com
<html>
<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.6.3</center>
</body>
</html>
# 当所有节点都宕机后,nginx向用户报告了502错误
2.5.5 同时开启所有节点
[root@lb01 conf]# curl www.tiandi.com
10.0.0.83 bbs
[root@lb01 conf]# curl www.tiandi.com
10.0.0.82 bbs
[root@lb01 conf]# curl www.tiandi.com
10.0.0.83 bbs
[root@lb01 conf]# curl www.tiandi.com
10.0.0.82 bbs
[root@lb01 conf]# curl www.tiandi.com
10.0.0.83 bbs
[root@lb01 conf]# curl www.tiandi.com
10.0.0.82 bbs
# 又会恢复到原来状态
第3章 nginx负载均衡配置实战
3.1 配置基于域名的虚拟主机web节点
以下操作是在web01上,web02做相同操作即可
3.1.1 nginx的配置文件如下:
[root@web01 ~]# cat /application/nginx/conf/nginx.conf
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
sendfile on;
keepalive_timeout 65;
server {
listen 80;
server_name bbs.tiandi.com;
location / {
root html/bbs;
index index.html index.htm;
}
access_log logs/access_bbs.log main;
}
server {
listen 80;
server_name www.tiandi.com;
location / {
root html/www;
index index.html index.htm;
}
access_log logs/access_www.log main;
}
}
以上配置文件配置了www.tinadi.com和bbs.tiandi.com两个虚拟主机
3.2 创建站点目录及对应的测试文件,命令如下:
[root@web01 html]# cat www/index.html
www.tiandi.com82
[root@web01 html]# cat bbs/index.html
bbs.tiandi.com82
3.3 检查语法并重启nginx服务
[root@web01 html]# ../sbin/nginx -t
nginx: the configuration file /application/nginx-1.6.3//conf/nginx.conf syntax is ok
nginx: configuration file /application/nginx-1.6.3//conf/nginx.conf test is successful
[root@web01 html]# ../sbin/nginx -s reload
3.4 把域名加入hosts解析,本机进行访问测试(web01和web02操作相同)
[root@web01 html]# echo '10.0.0.82 www.tiandi.com' >>/etc/hosts
[root@web01 html]# echo '10.0.0.82 bbs.tiandi.com' >>/etc/hosts
[root@web01 html]# tail -2 /etc/hosts
10.0.0.82 www.tiandi.com
10.0.0.82 bbs.tiandi.com
检查虚拟主机配置结果
10.0.0.82 web01上的测试结果如下:
[root@web01 html]# curl www.tiandi.com
www.tiandi.com82
[root@web01 html]# curl bbs.tiandi.com
bbs.tiandi.com82
10.0.0.83 web03上的测试结果如下:
[root@web02 html]# curl www.tiandi.com
www.tiandi.com83
[root@web02 html]# curl bbs.tiandi.com
bbs.tiandi.com83
第4章 nginx负载均衡反向代理实战
利用upstream定义一组www服务器池
4.1 nginx的实际配置如下:
[root@lb01 conf]# cat nginx.conf
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
upstream www_server_pools { 定义的服务器池
server 10.0.0.82:80 weight=1;
server 10.0.0.83:80 weight=1;
}
server {
listen 80;
server_name www.tiandi.com;
location / {
proxy_pass http://www_server_pools;
}
}
}
4.2 配置hosts解析到代理的ip或vip上,然后重新加载配置文件,访问测试
[root@lb01 conf]# tail -2 /etc/hosts
10.0.0.80 www.tiandi.com
10.0.0.80 bbs.tiandi.com
[root@lb01 conf]# ../sbin/nginx -s reload 重新加载nginx服务
[root@lb01 conf]# curl www.tiandi.com
bbs.tiandi.com82
[root@lb01 conf]# curl www.tiandi.com
bbs.tiandi.com83
[root@lb01 conf]# curl www.tiandi.com
bbs.tiandi.com82
[root@lb01 conf]# curl www.tiandi.com
bbs.tiandi.com83
从测试结果看,已经实现了反向代理,负载均衡功能,但是出来的结果并不是www.tiandi.comd 的字符串,而是bbs的信息。
想要实现出现www的信息其实很简单,就是在nginx代理的www服务虚拟主机配置里增加如下一行配置即可。
proxy_set_header Host $host;
4.3 整个nginx代理配置为:
[root@lb01 conf]# cat nginx.conf
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
upstream www_server_pools {
server 10.0.0.82:80 weight=1;
server 10.0.0.83:80 weight=1;
}
server {
listen 80;
server_name www.tiandi.com;
location / {
proxy_pass http://www_server_pools;
proxy_set_header Host $host;
}
}
}
4.4 重新加载服务,并测试
[root@lb01 conf]# ../sbin/nginx -s reload
[root@lb01 conf]# curl www.tiandi.com
www.tiandi.com82
[root@lb01 conf]# curl www.tiandi.com
www.tiandi.com83
这次访问的结果盒访问的域名完全对应上了,这样代理多虚拟主机的节点服务器就不会出问题了。
第5章 经过反向代理后的节点服务器记录用户ip企业案例
5.1 解决此问题同样是增加一行参数
proxy_set_header X-Forwarded-For $remote_addr;
这是反向代理时,及诶单服务器获取用户真实ip的必要功能配置
5.2 解决上述问题的整个nginx代理配置为:
[root@lb01 logs]# cat ../conf/nginx.conf
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
upstream www_server_pools {
server 10.0.0.82:80 weight=1;
server 10.0.0.83:80 weight=1;
}
server {
listen 80;
server_name www.tiandi.com;
location / {
proxy_pass http://www_server_pools;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
}
}
}
5.3 重新加载nginx反向代理服务
[root@lb01 logs]# ../sbin/nginx -s reload
特别注意:虽然反向代理这块已经配置好了,但是节点服务器需要的访问日志如果要记录用户的真实ip,还必须进行日志格式配置,这样才能把代理传过来的X-Forwarded-For头信息记录下来,具体配置为:(web01和web02配置相同)
[root@web01 html]# cat ../conf/nginx.conf
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"'; 记录客户端真实ip
sendfile on;
keepalive_timeout 65;
server {
listen 80;
server_name bbs.tiandi.com;
location / {
root html/bbs;
index index.html index.htm;
}
access_log logs/access_bbs.log main;
}
server {
listen 80;
server_name www.tiandi.com;
location / {
root html/www;
index index.html index.htm;
}
access_log logs/access_www.log main;
}
}