linux curl命令读取页面信息,使用Curl命令获取网站Web服务器类

Curl是一款用于在网络上传输数据的工具，支持HTTP, HTTPS, FTP, FTPS, TFTP, DICT,TELNET,LDAP等协议。通过curl你既可以将服务器的数据下载下来，也可以将本地的数据上传到服务器。curl的选项很多，大家可以参考curl manual。

最简单的用法，获取一个网站页面的源代码:

[root@linuxorz ~]# curl www.g.cn

301 Moved

301 Moved

The document has moved

here.

[root@linuxorz ~]# curl www.google.cn

言归正传，下面将告诉大家如何使用-Is选项来获取网站的web服务器类型。

-I :获取网站的HTTP-header信息

-s :静默模式。不显示进度及错误信息

--connect-timeout ：设置连接超时秒数

[root@linuxorz ~]#

curl -Is www.163.comHTTP/1.0 200 OK

Server: nginx

Date: Fri, 08 Jan 2010 14:40:23 GMT

Content-Type: text/html; charset=GBK

Vary: Accept-Encoding

Expires: Fri, 08 Jan 2010 14:41:43 GMT

Cache-Control: max-age=80

Vary: User-Agent

Vary: Accept

X-Cache: MISS from cache.163.com

Connection: close

首先创建一个site.txt文件，里面输入相应的网站。

[root@linuxorz ~]#

head site.txt

www.google.com

www.baidu.com

youa.baidu.com

post.baidu.com

hi.baidu.com

www.sogou.com

www.youdao.com

www.soso.com

www.sohu.com

www.sina.com

之后结合下面的curl.sh脚本，就可以获知site.txt中网站的服务器类型了。

#!/bin/sh

IIS=0

nginx=0

apache=0

other=0

if [ ! -f site.txt ]; then

echo "ERROR:site.txt NOT exists!"

exit 1

fi

total=`wc -l site.txt|awk '{print $1}'`

for website in `cat site.txt`

do

server=`curl -Is --connect-timeout 15 $website|awk -F":" '/^Server:/{print $2}'`

echo -e $website":" $server

if echo $server|grep -i "IIS">/dev/null

then IIS=`expr $IIS + 1`

elif echo $server|grep -i "Apache">/dev/null

then Apache=`expr $Apache + 1`

elif echo $server|grep -i "nginx">/dev/null

then nginx=`expr $nginx + 1`

else other=`expr $other + 1`

fi

done

echo "--------------------------------------------"

echo -e "Total/tApache/tIIS/tnginx/tother"

echo -e "$total/t$Apache/t$IIS/t$nginx/t$other"

echo -e "100%/t"`echo "scale=5;$Apache/$total*100"|bc|cut -c1-5`"%/t"`echo "scale=5;$IIS/$total*100"|bc|cut -c1-5`"%/t"`echo "scale=5;$nginx/$total*100"|bc|cut -c1-5`"%/t"`echo "scale=5;$other/$total*100"|bc|cut -c1-5`"%/t"

echo "--------------------------------------------"

exit 0

输出结果：

root@linuxorz ~]# bash curl.sh

www.google.com: gws

www.baidu.com: BWS/1.0

youa.baidu.com: Apache

post.baidu.com: apache 2.7.18.0

hi.baidu.com: apache 1.1.26.0

www.sogou.com: Apache

www.youdao.com: nginx

www.soso.com: SWS/1.0

www.sohu.com: SWS

www.sina.com: Apache/2.0.54 (Unix)

www.ifeng.com: DnionOS/1.0

www.alibaba.com: Apache/2.0.59 (Unix) mod_AliCookie(for apache2.x)/1.1 aliBeacon/1.0 mod_jk/1.2.25

www.163.com: nginx

www.126.com: Apache

www.263.com: Microsoft-IIS/6.0

www.tom.com: Apache

www.qq.com: nginx/0.6.39

qzone.qq.com: Apache

www.hotmail.com: Microsoft-IIS/6.0

www.wordpress.com: nginx

mail.163.com: Apache

mail.sina.com: Apache/2.2.9 (FreeBSD) mod_ssl/2.2.9 OpenSSL/0.9.7e-p1 PHP/5.2.6 with Suhosin-Patch

mail.tom.com: Apache/1.3.41 (Unix)

blog.sina.com: nginx/0.7.62

blog.sohu.com: nginx

blog.163.com: nginx

blog.hexun.com: Microsoft-IIS/6.0

www.hackbase.com: Microsoft-IIS/6.0

www.mydrivers.com: Microsoft-IIS/6.0

www.enet.com.cn: Apache

www.cnzz.com: Apache/2.2.4 (Unix) PHP/5.2.4

www.soufun.com: BIG-IP

www.chinaz.com: Microsoft-IIS/6.0

www.51la.com: nginx/0.6.35

www.download.com: Apache/2.2

www.godaddy.com: Microsoft-IIS/6.0

www.answers.com: Apache

www.sourceforge.net: BigIP

www.gmail.com: gws

www.msn.com: Microsoft-IIS/6.0

www.cctv.com: Apache

www.xinhuanet.com: Apache

www.renren.com: nginx/0.7.64

www.kaixin001.com: Apache

www.hao123.com: lighttpd

www.114la.com: Apache

www.douban.com: nginx

www.wikimedia.com: Apache/1.3 (Unix) mod_ssl/2.8.28 OpenSSL/0.9.8f AuthPG/1.3 FrontPage/5.0.2.2635

www.jiangmin.com: DnionOS/1.0

www.rising.com.cn: Microsoft-IIS/6.0

www.kaspersky.com: nginx/0.8.15

www.kingsoft.com: Apache/2.0.53

www.avast.com: Apache

www.360.cn: Apache

www.micropoint.com.cn: Apache

www.free-av.com: Apache

www.avg.com: Apache

www.hexun.com: nginx

www.eastmoney.com: Microsoft-IIS/6.0

www.chinahr.com: Microsoft-IIS/6.0

www.51job.com: Apache/2.2.8 (Unix) PHP/5.2.5

www.zhaopin.com: Apache/1.3.37 (Unix)

www.taobao.com: Apache

www.paipai.com: nginx

www.dangdang.com: nginx/0.6.37

www.youku.com: Apache

www.tudou.com: tws/0.1

www.ku6.com: Apache

6.cn: nginx/0.6.14

www.cntv.cn: Apache

media.17173.com: Apache

www.gougou.com: Apache/2.2.8 (Unix) mod_fastcgi/2.4.6

www.xunlei.com: nginx/0.7.62

www.verycd.com: Apache

www.pps.tv: PPStream

www.tianya.cn: Microsoft-IIS/5.0

dzh.mop.com: Resin/3.0.19

www.pconline.com.cn: nginx

www.zol.com.cn: Apache

www.pcpop.com: Microsoft-IIS/6.0

www.it168.com: Microsoft-IIS/6.0

www.newhua.com: Microsoft-IIS/6.0

www.skycn.com: Apache

www.crsky.com: Microsoft-IIS/6.0

www.csdn.net: CWS/1.0.64

www.chinaitlab.com: Microsoft-IIS/6.0

www.51.com: Apache

www.58.com: Microsoft-IIS/6.0

www.jiayuan.com: nginx/0.7.62

www.marry5.com: Apache

www.icbc.com.cn: Microsoft-IIS/6.0

www.abchina.com: Microsoft-IIS/7.5

www.ccb.com: Apache

www.boc.cn: Microsoft-IIS/7.0

www.bankcomm.com: IBM_HTTP_SERVER

www.ebay.com: Apache-Coyote/1.1

www.myspace.com: Microsoft-IIS/7.5

www.aol.com: Apache-Coyote/1.1

www.cnn.com: Apache

www.reuters.com: Apache

www.wto.org: Microsoft-IIS/6.0

www.who.org: nginx/0.6.35

www.gov.cn: Apache

www.court.gov.cn: Apache/2.0.52 (CentOS)

www.moe.edu.cn: Apache/2.0.54 (Unix) mod_jk2/2.0.4

www.mps.gov.cn: IBM_HTTP_Server/6.0.2.23 Apache/2.0.47 (Win32)

www.most.gov.cn: Microsoft-IIS/6.0

www.pbc.gov.cn: Microsoft-IIS/6.0

www.moh.gov.cn: Apache/2.0.59 (Win32)

www.ccnt.gov.cn: Microsoft-IIS/6.0

www.mof.gov.cn: Apache/2.0.59 HP-UX_Apache-based_Web_Server (Unix) DAV/2

www.mod.gov.cn: Apache

www.microsoft.com: Microsoft-IIS/7.5

www.microsoft.com.cn: Microsoft-IIS/7.0

www.centos.org: Apache/2.0.52 (CentOS)

www.redhat.com: Apache

www.ubuntu.com: Apache/2.2.8 (Ubuntu) mod_python/3.3.1 Python/2.5.2 PHP/5.2.4-2ubuntu5.9 with Suhosin-Patch mod_ssl/2.2.8 Ope

nSSL/0.9.8g

www.gentoo.org: Apache

www.debian.org: Apache/2.2.9 (Debian) mod_ssl/2.2.9 OpenSSL/0.9.8g

www.novell.com: Apache

www.slackware.com: Apache/1.3.27 (Unix) PHP/4.3.1

www.opensuse.org: Apache/2.2.10 (Linux/SUSE)

www.wikipedia.org: Apache

www.hp.com: Apache

www.dell.com: Microsoft-IIS/7.0

www.hitachi.com: Apache/2.2.11 (Unix) mod_ssl/2.2.11 OpenSSL/0.9.7d

www.toshiba.com: Sun-ONE-Web-Server/6.1

www.nokia.com: "Nokia"

www.sony.com: Apache

www.intel.com: IA Web Server/1.0

www.amd.com: AkamaiGHost

www.nvidia.com: Microsoft-IIS/6.0

www.sun.com: Sun-Java-System-Web-Server/7.0

www.ibm.com: IBM_HTTP_Server

--------------------------------------------

Total   Apache  IIS     nginx   other

134     64      30      19      21

100%    47.76%  22.38%  14.17%  15.67%

--------------------------------------------

统计的结果，似乎和实际使用情况还有些相近。是不是很简单