nginx的url_hash模块(第三方)
含义:按访问url的hash结果来分配请求,使每个url定向到同一个后端服务器,后端服务器为缓存时比较有效。
nginx本身是不支持url_hash的,需要打第三方的patch,下载地址:
http://anotherbug.blog.chinajavaworld.com/servlet/AttachmentServlet/download/123297-4298-2148/nginx_upstream_hash-0[1].2.tar.gz
See http://wiki.codemongers.com/NginxHttpUpstreamRequestHashModule for more details.
关于nginx的url_hash模块的解释就到此为止,下面我们来看看怎么用nginx的url_hash模块来提高squid的缓存效率的:
含义:按访问url的hash结果来分配请求,使每个url定向到同一个后端服务器,后端服务器为缓存时比较有效。
nginx本身是不支持url_hash的,需要打第三方的patch,下载地址:
http://anotherbug.blog.chinajavaworld.com/servlet/AttachmentServlet/download/123297-4298-2148/nginx_upstream_hash-0[1].2.tar.gz
See http://wiki.codemongers.com/NginxHttpUpstreamRequestHashModule for more details.
关于nginx的url_hash模块的解释就到此为止,下面我们来看看怎么用nginx的url_hash模块来提高squid的缓存效率的:
一.实验环境
四台服务器,主机名和IP分别为:
192.168.4.191----------nginx
192.168.4.188----------squid01
192.168.4.190----------squid02
192.168.4.189----------真实web
四台服务器,主机名和IP分别为:
192.168.4.191----------nginx
192.168.4.188----------squid01
192.168.4.190----------squid02
192.168.4.189----------真实web
二.nginx和squid的安装
1.nginx以及nginx的url_hash模块的安装
wget http://anotherbug.blog.chinajavaworld.com/servlet/AttachmentServlet/download/123297-4298-2148/nginx_upstream_hash-0[1].2.tar.gz
wget http://www.nginx.eu/download/sources/nginx-0.5.21.tar.gz
1) tar -zxvf nginx_upstream_hash-0.2.tar.gz -C /usr/local
mv nginx_upstream_hash-0.2 nginx
2) tar -zxvf nginx-0.5.21.tar.gz
cd nginx-0.5.21
patch -p0 < /usr/local/nginx/nginx-0.5.21.patch
./configure --with-http_stub_status_module --add-module=/usr/local/nginx/
make && make install
3) cd /usr/local/nginx/conf/
vi nginx.conf 内容如下:
1.nginx以及nginx的url_hash模块的安装
wget http://anotherbug.blog.chinajavaworld.com/servlet/AttachmentServlet/download/123297-4298-2148/nginx_upstream_hash-0[1].2.tar.gz
wget http://www.nginx.eu/download/sources/nginx-0.5.21.tar.gz
1) tar -zxvf nginx_upstream_hash-0.2.tar.gz -C /usr/local
mv nginx_upstream_hash-0.2 nginx
2) tar -zxvf nginx-0.5.21.tar.gz
cd nginx-0.5.21
patch -p0 < /usr/local/nginx/nginx-0.5.21.patch
./configure --with-http_stub_status_module --add-module=/usr/local/nginx/
make && make install
3) cd /usr/local/nginx/conf/
vi nginx.conf 内容如下:
user nobody nobody;
worker_processes 4;
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#pid logs/nginx.pid;
worker_rlimit_nofile 51200;
events
{
use epoll;
worker_connections 51200;
}
worker_processes 4;
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#pid logs/nginx.pid;
worker_rlimit_nofile 51200;
events
{
use epoll;
worker_connections 51200;
}
http
{
# include mime.types;
default_type application/octet-stream;
keepalive_timeout 20;
tcp_nodelay on;
upstream www.jialisong.com {
server 192.168.4.188 ;
server 192.168.4.190 ;
hash $request_uri;
}
server
{
listen 80;
server_name www.jialisong.com;
location / {
proxy_pass http://www.jialisong.com;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
}
log_format main '$remote_addr - $remote_user [$time_local] $request '
'"$status" $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /usr/local/nginx/logs/access_log ;
location /NginxStatus {
stub_status on;
access_log on;
auth_basic "NginxStatus";
# auth_basic_user_file htpasswd;
{
# include mime.types;
default_type application/octet-stream;
keepalive_timeout 20;
tcp_nodelay on;
upstream www.jialisong.com {
server 192.168.4.188 ;
server 192.168.4.190 ;
hash $request_uri;
}
server
{
listen 80;
server_name www.jialisong.com;
location / {
proxy_pass http://www.jialisong.com;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
}
log_format main '$remote_addr - $remote_user [$time_local] $request '
'"$status" $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /usr/local/nginx/logs/access_log ;
location /NginxStatus {
stub_status on;
access_log on;
auth_basic "NginxStatus";
# auth_basic_user_file htpasswd;
}
}
}
}
}
4) /usr/local/nginx/sbin/nginx -t 测试nginx的配置文件
/usr/local/nginx/sbin/nginx 启动nginx
/usr/local/nginx/sbin/nginx 启动nginx
2.squid服务器的安装
1)tar -jxvf squid-2.7.STABLE5.tar.bz2
cd squid-2.7.STABLE5
./configure --prefix=/usr/local/squid --enable-dlmalloc --with-pthreads --enable-poll --disable-internal-dns --enable-stacktrace --enable-removal-policies="heap,lru" --enable-delay-pools --enable-storeio="aufs,coss,diskd,ufs"
make && make install
useradd -s /sbin/login squid
chown -R squid:squid /usr/local/squid/var/
mkdir /tmp1
chmod -R 777 /tmp1
2) cd /usr/local/squid/etc/
vi squid.conf
内容如下:
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl Localhost src 127.0.0.1 192.168.4.188 192.168.4.191
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80
acl CONNECT method CONNECT
#acl SNMP snmp_community public
#acl myip dst 192.168.4.191
#http_access deny !myip
http_access allow all
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
#snmp_access allow SNMP Localhost
#acl myip dst 192.168.30.93
#http_access deny !myip
1)tar -jxvf squid-2.7.STABLE5.tar.bz2
cd squid-2.7.STABLE5
./configure --prefix=/usr/local/squid --enable-dlmalloc --with-pthreads --enable-poll --disable-internal-dns --enable-stacktrace --enable-removal-policies="heap,lru" --enable-delay-pools --enable-storeio="aufs,coss,diskd,ufs"
make && make install
useradd -s /sbin/login squid
chown -R squid:squid /usr/local/squid/var/
mkdir /tmp1
chmod -R 777 /tmp1
2) cd /usr/local/squid/etc/
vi squid.conf
内容如下:
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl Localhost src 127.0.0.1 192.168.4.188 192.168.4.191
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80
acl CONNECT method CONNECT
#acl SNMP snmp_community public
#acl myip dst 192.168.4.191
#http_access deny !myip
http_access allow all
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
#snmp_access allow SNMP Localhost
#acl myip dst 192.168.30.93
#http_access deny !myip
#acl Manager proto cache_object
#acl Localhost src 127.0.0.1 192.168.30.93
#http_access allow Manager Localhost
#http_access deny Manager
#acl Localhost src 127.0.0.1 192.168.30.93
#http_access allow Manager Localhost
#http_access deny Manager
#acl Safe_ports port 81
#http_access deny !Safe_ports
#http_access allow all
#http_access deny !Safe_ports
#http_access allow all
#acl OverConnLimit maxconn 16
#http_access deny OverConnLimit
http_access deny all
#http_access deny OverConnLimit
http_access deny all
icp_access allow all
http_port 192.168.4.188:80 vhost vport
cache_peer 192.168.4.189 parent 80 0 no-digest no-query originserver name=www
cache_peer_domain www www.jialisong.com
cache_peer_domain www www.jialisong.com
hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin .php .cgi .avi .wmv .rm .ram .mpg .mpeg .zip .exe
cache deny QUERY
cache_mem 300 MB
maximum_object_size_in_memory 4096 KB
cache_dir ufs /tmp1 2048 16 512
ie_refresh on
cache_swap_low 90
cache_swap_high 95
#ipcache_size 4096
#ipcache_low 90
#ipcache_high 95
fqdncache_size 1024
logformat combined %>a %ui %un [%tl] "%rm %ru HTTP/%rv" %Hs %<st "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh
#LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
access_log none
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log squid
cache_store_log none
emulate_httpd_log on
request_header_max_size 100 KB
url_rewrite_host_header off
acl QUERY urlpath_regex cgi-bin .php .cgi .avi .wmv .rm .ram .mpg .mpeg .zip .exe
cache deny QUERY
cache_mem 300 MB
maximum_object_size_in_memory 4096 KB
cache_dir ufs /tmp1 2048 16 512
ie_refresh on
cache_swap_low 90
cache_swap_high 95
#ipcache_size 4096
#ipcache_low 90
#ipcache_high 95
fqdncache_size 1024
logformat combined %>a %ui %un [%tl] "%rm %ru HTTP/%rv" %Hs %<st "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh
#LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
access_log none
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log squid
cache_store_log none
emulate_httpd_log on
request_header_max_size 100 KB
url_rewrite_host_header off
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 4320
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 4320
refresh_pattern -i .gif$ 1440 50% 2880 ignore-reload
refresh_pattern -i .jpg$ 1440 50% 2880 ignore-reload
refresh_pattern -i .png$ 1440 50% 2880 ignore-reload
refresh_pattern -i .jpg$ 1440 50% 2880 ignore-reload
refresh_pattern -i .png$ 1440 50% 2880 ignore-reload
refresh_pattern -i .mp3$ 1440 50% 2880 ignore-reload
refresh_pattern -i .wmv$ 1440 50% 2880 ignore-reload
refresh_pattern -i .rm$ 1440 50% 2880 ignore-reload
refresh_pattern -i .swf$ 1440 50% 2880 ignore-reload
refresh_pattern -i .mpeg$ 1440 50% 2880 ignore-reload
refresh_pattern -i .wma$ 1440 50% 2880 ignore-reload
refresh_pattern -i .bmp$ 1440 50% 2880 ignore-reload
refresh_pattern -i .txt$ 1440 50% 2880 ignore-reload
refresh_pattern -i .wmv$ 1440 50% 2880 ignore-reload
refresh_pattern -i .rm$ 1440 50% 2880 ignore-reload
refresh_pattern -i .swf$ 1440 50% 2880 ignore-reload
refresh_pattern -i .mpeg$ 1440 50% 2880 ignore-reload
refresh_pattern -i .wma$ 1440 50% 2880 ignore-reload
refresh_pattern -i .bmp$ 1440 50% 2880 ignore-reload
refresh_pattern -i .txt$ 1440 50% 2880 ignore-reload
refresh_pattern -i .css$ 10 50% 60 reload-into-ims
refresh_pattern -i .js$ 10 50% 60 reload-into-ims
refresh_pattern -i .xml$ 10 50% 30 reload-into-imsfresh_pattern -i .html$ 720 50% 1440 reload-into-ims
refresh_pattern -i .js$ 10 50% 60 reload-into-ims
refresh_pattern -i .xml$ 10 50% 30 reload-into-imsfresh_pattern -i .html$ 720 50% 1440 reload-into-ims
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
cache_effective_user squid
cache_effective_group squid
visible_hostname pic.mytophome.com
cache_mgr molecar@gmail.com
httpd_suppress_version_string on
cache_vary on
icp_port 0
error_directory /usr/local/squid/share/errors/English
via off
forwarded_for on
tcp_recv_bufsize 65535 bytes
coredump_dir /usr/local/squid/var/cache
max_filedesc 20480
3)启动squid
/usr/local/squid/sbin/squid -z
/usr/local/squid/sbin/squid -NCd1
/usr/local/squid/sbin/squid -f /usr/local/squid/etc/squid.conf
两台squid的安装配置一样只是在squid.conf配置文件做稍微的修改即可
/usr/local/squid/sbin/squid -z
/usr/local/squid/sbin/squid -NCd1
/usr/local/squid/sbin/squid -f /usr/local/squid/etc/squid.conf
两台squid的安装配置一样只是在squid.conf配置文件做稍微的修改即可
三.测试
1)修改自己电脑的hosts文件,把 www.jialisong.com指向到nginx服务器的ip上(目的在于使 www.jialisong.com和192.168.4.191相对应)
2)打开两台squid的日志,用tail -f的方式,然后访问 www.jialisong.com,可以看到两边的日志的内容是不一样的,从其中一台squid的日志中找一条静态文件的url,
通过ie 打开,可以发现,这个url的访问只投到同一台squid上,如此实现了,squid缓存的内容不同,从而增加了squid的缓存容量
当然,如果中间只使用一台nginx,自然这台会成为单点而造成隐患,所以可是使用heartbeat做成双机,增加系统的可靠性。
1)修改自己电脑的hosts文件,把 www.jialisong.com指向到nginx服务器的ip上(目的在于使 www.jialisong.com和192.168.4.191相对应)
2)打开两台squid的日志,用tail -f的方式,然后访问 www.jialisong.com,可以看到两边的日志的内容是不一样的,从其中一台squid的日志中找一条静态文件的url,
通过ie 打开,可以发现,这个url的访问只投到同一台squid上,如此实现了,squid缓存的内容不同,从而增加了squid的缓存容量
当然,如果中间只使用一台nginx,自然这台会成为单点而造成隐患,所以可是使用heartbeat做成双机,增加系统的可靠性。
转载于:https://blog.51cto.com/jlsfwq/172654