linux 打开网页 用curl_linux 使用curl命令访问url并模拟cookie

1.目录

linux下通过命令访问url的方式有多种，主要如下

2.1.elinks

elinks – lynx-like替代角色模式WWW的浏览器

例如： elinks –dump http://www.baidu.com

2.1.2.wget

这个会将访问的首页下载到本地

[root@el5-mq2~]#wget http://www.baidu.com

3.3.curl

curl会显示出源码

curl http://www.baidu.com/index.html

4.4.lynx

lynx http://www.baidu.com

5.5.curl使用实践

现在有个需求，因为服务器在收集访问数据，抓取cookie中的value，模拟url访问时需要带上cookie参数，curl命令刚好能完成这个功能。

首先查看帮助：

curl-h

-b/–cookieCookiestringorfile to read cookiesfrom(H)

-c/–cookie-jarWritecookies tothisfile after operation(H)

–create-dirsCreatenecessarylocaldirectory hierarchy

–crlfConvertLF to CRLFinupload

–crlfileGeta CRL listinPEM formatfromthe given file

可以使用-b参数来完成，具体使用如下：

curl–b“key1=val1;key2=val2;”

或直接使用文件

curl-b./cookie.txt

编写测试实例：

curl-b“user_trace_token=20150518150621-02994ed9a0fb42d1906a56258e072fc4;LGUID=20150515135257-a33a769c-fac6-11e4-91ce-5254005c3644”http://10.10.26.164:1235/click?v=1&logtype=deliver&position=home_hot-0&orderid=10197777&userid=1942556&positionid=148&url=http%3a%2f%2fwww.lagou.com%2fjobs%2f317000.html%3fsource%3dhome_hot%26i%3dhome_hot-5&fromsite=http%3a%2f%2fwww.lagou.com%2fzhaopin%2fAndroid%3flabelWords%3dlabel%26utm_source%3dAD__baidu_pinzhuan%26utm_medium%3dsem%26utm_campaign%3dSEM&optime=2015-06-15_20:00:00

发现这样还是不可以，url附带的参数取不到。使用-d 参数传递url参数，使用-G 把请求方式配置为GET就OK了，如下：

curl-b“user_trace_token=20150518150621-02994ed9a0fb42d1906a56258e072fc4;LGUID=20150515135257-a33a769c-fac6-11e4-91ce-5254005c3644;LGSID=20150518150621-02994ed9a0fb42d1906a56258e072fc4;LGRID=20150617230732-4ea87972-1580-11e5-9a88-000c29653e90;”-d“v=1&logtype=deliver&position=i_home-1&orderid=10197777&userid=1942556&positionid=148&url=http%3a%2f%2fwww.lagou.com%2fjobs%2f317000.html%3fsource%3dhome_hot%26i%3dhome_hot-5&fromsite=http%3a%2f%2fwww.lagou.com%2fzhaopin%2fAndroid%3flabelWords%3dlabel%26utm_source%3dAD__baidu_pinzhuan%26utm_medium%3dsem%26utm_campaign%3dSEM&optime=2015-06-15_20:00:00”-G http://10.10.26.164:1235/click

想要获得response返回的cookie怎么办，使用’-c’参数，同时可以使用-b filename用文件方式表示cookie，配合-c使用更方便

可以先用-c 命令生成一个cookie文件作为模板，再修改这个文件作为-b 参数的文件名。

使用如下：

curl-b c1.txt-c c2.txt-d“v=1&_v=j31&a=406405635&t=pageview&_s=1&dr=http%3a%2f%2fwww.sogou.com%2ftuguang&dl=http%3A%2F%2Fwww.lagou.com%2F%3futm_source%3dad_sougou_pingzhuan&ul=zh-cn&de=UTF-8&dt=%E6%8B%89%E5%8B%BE%E7%BD%91-%E6%9C%80%E4%B8%93%E4%B8%9A%E7%9A%84%E4%BA%92%E8%81%94%E7%BD%91%E6%8B%9B%E8%81%98%E5%B9%B3%E5%8F%B0&sd=24-bit&sr=1600×900&vp=1583×291&je=1&fl=18.0%20r0&_u=MACAAAQBK~&jid=&cid=1312768212.1431333683&tid=UA-41268416-1&z=1204746223”-G http://192.168.52.130:1234/collect

生成的c2.txt内容如下：

# Netscape HTTP Cookie File

# http://curl.haxx.se/docs/http-cookies.html

# This file was generated by libcurl! Edit at your own risk.192.168.52.130 FALSE / FALSE 1757574737 user_trace_token 20150914151217-eedd019e-5aaf-11e5-8a69-000c29653e90

192.168.52.130FALSE/FALSE1442217595LGSID20150914152955-652a13c5-5ab2-11e5-846d-000c29653e90

192.168.52.130FALSE/FALSE1442217595PRE_UTM

192.168.52.130FALSE/FALSE1442217595PRE_HOST www.huxiu.com

192.168.52.130FALSE/FALSE1442217595PRE_SITE http%3A%2F%2Fwww.huxiu.com%2Ftuguang

192.168.52.130FALSE/FALSE1442217595PRE_LAND http%3A%2F%2Fwww.lagou.com%2F%3F

192.168.52.130FALSE/FALSE0LGRID20150914152955-652a1630-5ab2-11e5-846d-000c29653e90

192.168.52.130FALSE/FALSE1757574737LGUID20150914151217-eedd0624-5aaf-11e5-8a69-000c29653e90

直接请求：

curl-X POST-d'msg={"cmd":"ticker","userid":18252928,"platform":"mqtt"}'http://msg.live/msg-server/apiRequest