这几天闲着没事从老男孩老师看到一个淘宝网的框架图,挺感兴趣的,cdn和集群线上的架构我都做过,但是没有接触过这么大的环境,先简单的实现看看, 当然了真正的淘宝架构肯定不能像我这样的,但是自己过过实验瘾也挺爽的。
陆续的把脚本贴出来。。。
脚本有不严谨的地方,请大家指出。。。。。
脚本的ip貌似和图上都对不上,自己修改和增加吧~~~~~~~~~
说实话,lvs配置是最简单,没什么好配置的,集群环境我用lvs较少,因为没有正则的功能,当然了他作为4层的东西,优势在于大流量的承载转发。
- mkdir/usr/local/src/lvs
- cd/usr/local/src/lvs
- wgethttp://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz
- wgethttp://www.keepalived.org/software/keepalived-1.1.15.tar.gz
- lsmod|grepip_vs
- uname-r
- ln-s/usr/src/kernels/$(uname-r)/usr/src/linux
- tarzxvfipvsadm-1.24.tar.gz
- cdipvsadm-1.24
- make&&makeinstall
- tarzxvfkeepalived-1.1.15.tar.gz
- cdkeepalived-1.1.15
- ./configure&&make&&makeinstall
- cp/usr/local/etc/rc.d/init.d/keepalived/etc/rc.d/init.d/
- cp/usr/local/etc/sysconfig/keepalived/etc/sysconfig/
- mkdir/etc/keepalived
- cp/usr/local/etc/keepalived/keepalived.conf/etc/keepalived/
- cp/usr/local/sbin/keepalived/usr/sbin/
- #youcanservicekeepalivedstart|stop
- #master
- cat>>/usr/local/etc/keepalived/keepalived.conf<<EOF
- !ConfigurationFileforkeepalived
- global_defs{
- notification_email{
- rfyiamcool@163.com
- }
- notification_email_fromAlexandre.Cassen@firewall.loc
- smtp_server127.0.0.1
- router_idLVS_DEVEL
- }
- vrrp_instanceVI_1{
- stateMASTER # other backup
- interfaceeth0
- virtual_router_id51
- priority100 # other 90
- advert_int1
- authentication{
- auth_typePASS
- auth_pass1111
- }
- virtual_ipaddress{
- 10.10.10.88
- }
- }
- virtual_server10.10.10.8880{
- delay_loop6
- lb_algorr
- lb_kindDR
- persistence_timeout50
- protocolTCP
- real_server10.10.10.2180{
- weight3
- TCP_CHECK{
- connect_timeout10
- nb_get_retry3
- delay_before_retry3
- connect_port80
- }
- }
- real_server10.10.10.2280{
- weight3
- TCP_CHECK{
- connect_timeout10
- nb_get_retry3
- delay_before_retry3
- connect_port80
- }
- }
- }
- EOF
- servicekeepalivedstart
咱们先把二层的haproxy搞定,ip什么的大家自己改吧。
- #!/bin/bash
- cd/usr/local/src/
- wgethttp://haproxy.1wt.eu/download/1.4/src/haproxy-1.4.8.tar.gz
- tarzxfhaproxy-1.4.8.tar.gz
- cdhaproxy-1.4.8
- uname-a
- makeTARGET=linux26PREFIX=/usr/local/haproxy
- makeinstallPREFIX=/usr/local/haproxy
- cat>>/usr/local/haproxy/haproxy.cfg<<EOF
- global
- log127.0.0.1local0###全局日志
- maxconn4096###最大连接数
- chroot/usr/local/haproxy
- uid501###用户ID
- gid501###组ID
- daemon###后台运行
- nbproc1###创建进程数
- pidfile/usr/local/haproxy/haproxy.pid###pid文件
- defaults
- log127.0.0.1local3
- modehttp###支持的模式
- optionhttplog###日志格式
- optionhttpclose###请求完成后关闭http通道
- optiondontlognull
- optionforwardfor###apache日志转发
- optionredispatch
- retries2###重连次数
- maxconn2000
- balanceroundrobin###算法类型
- statsuri/haproxy-stats###状态统计页面
- #statsauthadmin:admin###状态统计页面用户名密码,可选
- contimeout5000###连接超时
- clitimeout50000###客户端超时
- srvtimeout50000###服务器超时
- listenproxy*:80###访问地址及端口
- optionhttpchkHEAD/index.htmlHTTP/1.0###健康检查页面
- serverweb210.10.10.30:88cookieapp1inst2checkinter2000rise2fall5
- serverweb210.10.10.31:88cookieapp1inst2checkinter2000rise2fall5
- serverweb210.10.10.32:88cookieapp1inst2checkinter2000rise2fall5
- serverweb210.10.10.33:88cookieapp1inst2checkinter2000rise2fall5
- serverweb210.10.10.34:88cookieapp1inst2checkinter2000rise2fall5
- serverweb210.10.10.35:88cookieapp1inst2checkinter2000rise2fall5
- EOF
- cat>>/etc/init.d/haproxy<<EOF
- #!/bin/sh
- set-e
- PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/haproxy/sbin
- PROGDIR=/usr/local/haproxy
- PROGNAME=haproxy
- DAEMON=\$PROGDIR/sbin/\$PROGNAME
- CONFIG=\$PROGDIR/\$PROGNAME.cfg
- PIDFILE=\$PROGDIR/\$PROGNAME.pid
- DESC="HAProxydaemon"
- SCRIPTNAME=/etc/init.d/\$PROGNAME
- #Gracefullyexitifthepackagehasbeenremoved.
- test-x\$DAEMON||exit0
- start()
- {
- echo-n"Starting\$DESC:\$PROGNAME"
- \$DAEMON-f\$CONFIG
- echo"."
- }
- stop()
- {
- echo-n"Stopping\$DESC:\$PROGNAME"
- haproxy_pid=cat/usr/local/haproxy/haproxy.pid
- kill\$haproxy_pid
- echo"."
- }
- restart()
- {
- echo-n"Restarting\$DESC:\$PROGNAME"
- \$DAEMON-f\$CONFIG-p\$PIDFILE-sf\$(cat\$PIDFILE)
- echo"."
- }
- case"\$1"in
- start)
- start
- ;;
- stop)
- stop
- ;;
- restart)
- restart
- ;;
- *)
- echo"Usage:\$SCRIPTNAME{start|stop|restart}">&2
- exit1
- ;;
- esac
- exit0
- EOF
- chmod+x/etc/rc.d/init.d/haproxy
- chkconfig--addhaproxy
- chmod777/usr/local/haproxy/haproxy.pid
- sed-i'/SYSLOGD_OPTIONS/c\SYSLOGD_OPTIONS="-r-m0"'/etc/sysconfig/syslog
- echo"local3.*/var/log/haproxy.log"/etc/syslog.conf
- echo"local0.*/var/log/haproxy.log"/etc/syslog.conf
- servicesyslogrestart
- #启动haproxy
- #/usr/local/haproxy/sbin/haproxy-f/usr/local/haproxy/haproxy.cfg
- #重启haproxy
- #/usr/local/haproxy/sbin/haproxy-f/usr/local/haproxy/haproxy.cfg-st`cat/usr/local/haproxy/haproxy.pid`
- #停止haproxy
- #killallhaproxy
- #servicehaproxystartrestartstop
haproxy是识别主机名的判断的 主机名的判断的例子格式如下:
- aclurl_aaahdr_dom(host)www.aaa.com
- aclurl_bbbhdr_dom(host)www.bbb.com
- acltm_policyhdr_dom(host)-itrade.gemini.taobao.net
- acldenali_policyhdr_reg(host)-i^(my.gemini.taobao.net|auction1.gemini.taobao.net)$
- aclpath_url163path_beg-i/163
- aclpath_url_bbbpath_beg-i/
- use_backendaaaifurl_aaa
- use_backendbbbifurl_bbb
- use_backendurl163ifurl_aaapath_url163
- backendurl163
- modehttp
- balanceroundrobin
- optionhttpchkGET/163/test.jsp
- serverurl16310.10.10.31:8080cookie1checkinter2000rise3fall3maxconn50000
- backendaaa
- modehttp
- balanceroundrobin
- optionhttpchkGET/test.jsp
- sreverapp_808010.10.10.32:8080cookie1checkinter1500rise3fall3maxconn50000
- backendbbb
- modehttp
- balanceroundrobin
- optionhttpchkGET/test.jsp
- sreverapp_808010.10.10.33:8090cookie1checkinter1500rise3fall3maxconn50000
haproxy端还要做lvs客户端模式,绑定回环口。
- #!/bin/bash
- SNS_VIP=10.10.10.88
- source/etc/rc.d/init.d/functions
- case"$1"in
- start)
- ifconfiglo:0$SNS_VIPnetmask255.255.255.255broadcast$SNS_VIP
- /sbin/routeadd-host$SNS_VIPdevlo:0
- echo"1">/proc/sys/net/ipv4/conf/lo/arp_ignore
- echo"2">/proc/sys/net/ipv4/conf/lo/arp_announce
- echo"1">/proc/sys/net/ipv4/conf/all/arp_ignore
- echo"2">/proc/sys/net/ipv4/conf/all/arp_announce
- echo"RealServerStartOK"
- ;;
- stop)
- ifconfiglo:0down
- routedel$SNS_VIP>/dev/null2>&1
- echo"0">/proc/sys/net/ipv4/conf/lo/arp_ignore
- echo"0">/proc/sys/net/ipv4/conf/lo/arp_announce
- echo"0">/proc/sys/net/ipv4/conf/all/arp_ignore
- echo"0">/proc/sys/net/ipv4/conf/all/arp_announce
- echo"RealServerStoped"
- ;;
- *)
- echo"Usage:$0{start|stop}"
- exit1
- esac
- exit0
下面是squid的设置
- #!/bin/bash
- wgethttp://www.squid-cache.org/Versions/v2/2.6/squid-2.6.STABLE6.tar.bz2
- tarjxvfsquid-2.6.STABLE6.tar.bz2
- ./configure--prefix=/usr/local/squid\
- --enable-async-io=320\
- --enable-storeio="aufs,diskd,ufs"\
- --enable-useragent-log\
- --enable-referer-log\
- --enable-kill-parent-hack\
- --enable-forward-log\
- --enable-snmp\
- --enable-cache-digests\
- --enable-default-err-language=Simplify_Chinese\
- --enable-epoll\
- --enable-removal-policies="heap,lru"\
- --enable-large-cache-files\
- --disable-internal-dns\
- --enable-x-accelerator-vary\
- --enable-follow-x-forwarded-for\
- --disable-ident-lookups\
- --with-large-files\
- --with-filedescriptors=65536
- cat>>/usr/local/squid/etc/squid.conf<<EOF
- visible_hostnamecache1.taobao.com
- http_port192.168.1.44:80vhostvport
- icp_port0
- cache_mem512MB
- cache_swap_low90
- cache_swap_high95
- maximum_object_size20000KB
- maximum_object_size_in_memory4096KB
- cache_dirufs/tmp1300032256
- cache_store_lognone
- emulate_httpd_logon
- efresh_pattern^ftp:144020%10080
- refresh_pattern^gopher:14400%1440
- refresh_pattern.020%4320
- negative_ttl5minutes
- positive_dns_ttl6hours
- negative_dns_ttl1minute
- connect_timeout1minute
- read_timeout15minutes
- request_timeout5minutes
- client_lifetime1day
- half_closed_clientson
- maximum_single_addr_tries1
- uri_whitespacestrip
- ie_refreshoff
- logformatcombined%>a%ui%un[%tl]"%rm%ruHTTP/%rv"%Hs%<st"%{Referer}>h""%{User-Agent}>h"%Ss:%Sh
- pid_filename/var/log/squid/squid.pid
- cache_log/var/log/squid/cache.log
- access_log/var/log/squid/access.logcombined
- aclallsrc0.0.0.0/0.0.0.0
- aclQUERYurlpath_regexcgi-bin.php.cgi.avi.wmv.rm.ram.mpg.mpeg.zip.exe
- cachedenyQUERY
- aclpicurlurl_regex-i\.bmp$\.png$\.jpg$\.gif$\.jpeg$
- aclmystie1referer_regex-iaaa
- http_accessallowmystie1picurl
- aclmystie2referer_regex-ibbb
- http_accessallowmystie2picurl
- aclnullrefreferer_regex-i^$
- http_accessallownullref
- aclhasrefreferer_regex-i.+
- http_accessdenyhasrefpicurl
- cache_peer192.168.1.7parent800no-queryoriginserverno-digestname=all
- cache_peer_domainall*.taobao.com
- cache_effective_usernobody
- cache_effective_groupnobody
- acllocalhostsrc127.0.0.1
- aclmy_other_proxysrcdomain.a.com
- follow_x_forwarded_forallowlocalhost
- follow_x_forwarded_forallowall#允许转发headip头
- acl_uses_indirect_clienton#只有2.6才有这这个个参数
- delay_pool_uses_indirect_clienton#只有2.6才有这这个个参数
- log_uses_indirect_clienton#只有2.6才有这这个个参数
- #refresh_pattern^ftp:6020%10080
- #refresh_pattern^gopher:600%1440
- #refresh_pattern^gopher:600%1440
- #refresh_pattern.020%1440
- refresh_pattern-i\.js$144050%2880 reload-into-ims ignore-reload ignore-no-cache ignore-auth ignore-private
- refresh_pattern-i\.html$72050%1440 reload-into-ims ignore-reload ignore-no-cache ignore-auth ignore-private
- refresh_pattern-i\.jpg$144090%2880 reload-into-ims ignore-reload ignore-no-cache ignore-auth ignore-private
- refresh_pattern-i\.gif$144090%2880 reload-into-ims ignore-reload ignore-no-cache ignore-auth ignore-private
- refresh_pattern-i\.swf$144090%2880 reload-into-ims ignore-reload ignore-no-cache ignore-auth ignore-private
- refresh_pattern-i\.jpg$144050%2880 reload-into-ims ignore-reload ignore-no-cache ignore-auth ignore-private
- refresh_pattern-i\.png$144050%2880 reload-into-ims ignore-reload ignore-no-cache ignore-auth ignore-private
- refresh_pattern-i\.bmp$144050%2880 reload-into-ims ignore-reload ignore-no-cache ignore-auth ignore-private
- refresh_pattern-i\.doc$144050%2880 reload-into-ims ignore-reload ignore-no-cache ignore-auth ignore-private
- refresh_pattern-i\.ppt$144050%2880 reload-into-ims ignore-reload ignore-no-cache ignore-auth ignore-private
- refresh_pattern-i\.xls$144050%2880 reload-into-ims ignore-reload ignore-no-cache ignore-auth ignore-private
- refresh_pattern-i\.pdf$144050%2880 reload-into-ims ignore-reload ignore-no-cache ignore-auth ignore-private
- refresh_pattern-i\.rar$144050%2880 reload-into-ims ignore-reload ignore-no-cache ignore-auth ignore-private
- refresh_pattern-i\.zip$144050%2880 reload-into-ims ignore-reload ignore-no-cache ignore-auth ignore-private
- refresh_pattern-i\.txt$144050%2880 reload-into-ims ignore-reload ignore-no-cache ignore-auth ignore-private
- EOF
- #建立缓存和日志目录,并改变权限使squid能写入
- mkdir/tmp1
- mkdir/var/log/squid
- chown-Rnobody:nobody/tmp1
- chmod666/tmp1
- chown-Rnobody:nobody/var/log/squid
- #首次运行squid要先建立缓存
- /usr/local/squid/sbin/squid-z
- #启动squid
- echo"65535">/proc/sys/fs/file-max
- ulimit-HSn65535
- /usr/local/squid/sbin/squid
缓存的清理脚本是从洒哥那里搞到的
只是根据洒哥的脚本很简单的延伸了下,以前那个分享的脚本可以去除域名和特定的文件格式,然后我就想了能不能去除一个网址的所有jpg 或者是 www.92hezu.com/123/bbb/ 这样的。 原来多家几个后缀,用grep过滤就ok了
qingli.sh www.xiuxiukan.com
qingli.sh jpg
qingli.sh xiuxiukan.com 123 bbb jpg
- #!/bin/sh
- squidcache_path="/squidcache"
- squidclient_path="/home/local/squid/bin/squidclient"
- #grep-a-r$1$squidcache_path/*|grep"http:"|awk-F'http:''{print"http:"$2;}'|awk-F\''{print$1}'>cache.txt
- if[["$1"=="swf"||"$1"=="png"||"$1"=="jpg"||"$1"=="ico"||"$1"=="gif"||"$1"=="css"||"$1"=="js"||"$1"=="html"||"$1"=="shtml"||"$1"=="htm"]];then
- grep-a-r.$1$squidcache_path/*|strings|grep"http:"|awk-F'http:''{print"http:"$2;}'|awk-F\''{print$1}'|grep"$1$"|uniq>cache.txt
- else
- grep-a-r$1$squidcache_path/*|strings|grep"http:"|grep$2$|grep$3$|grep$4$|grep$5$|grep$6$|awk-F'http:''{print"http:"$2;}'|awk-F\''{print$1}'|uniq>cache.txt
- fi
- sed-i"s/\";$//g"cache.txt
- catcache.txt|whilereadLINE
- do
- $squidclient_path-p80-mPURGE$LINE
- done