curl使用

最新推荐文章于 2024-06-25 18:52:24 发布

耿小渣

最新推荐文章于 2024-06-25 18:52:24 发布

阅读量2.2k

点赞数

分类专栏： linux 系统

本文链接：https://blog.csdn.net/u014608280/article/details/107022739

版权

系统同时被 2 个专栏收录

9 篇文章 0 订阅

订阅专栏

linux

6 篇文章 0 订阅

订阅专栏

curl 是常用的命令行工具，用来请求 Web 服务器。它的名字就是客户端（client）的 URL 工具的意思。

curl  https://www.baidu.com
<!DOCTYPE html>
<!--STATUS OK--><html> <head><meta http-equiv=content-type content=text/html;charset=utf-8><meta http-equiv=X-UA-Compatible content=IE=Edge><meta content=always name=referrer><link rel=stylesheet type=text/css href=https://ss1.bdstatic.com/5eN1bjq8AAUYm2zgoY3K/r/www/cache/bdorz/baidu.min.css><title>百度一下，你就知道</title></head> <body link=#0000cc> <div id=wrapper> <div id=head> <div class=head_wrapper> <div class=s_form> <div class=s_form_wrapper> <div id=lg> <img hidefocus=true src=//www.baidu.com/img/bd_logo1.png width=270 height=129> </div> <form id=form name=f action=//www.baidu.com/s class=fm> <input type=hidden name=bdorz_come value=1> <input type=hidden name=ie value=utf-8> <input type=hidden name=f value=8> <input type=hidden name=rsv_bp value=1> <input type=hidden name=rsv_idx value=1> <input type=hidden name=tn value=baidu><span class="bg s_ipt_wr"><input id=kw name=wd class=s_ipt value maxlength=255 autocomplete=off autofocus=autofocus></span><span class="bg s_btn_wr"><input type=submit id=su value=百度一下 class="bg s_btn" autofocus></span> </form> </div> </div> <div id=u1> <a href=http://news.baidu.com name=tj_trnews class=mnav>新闻</a> <a href=https://www.hao123.com name=tj_trhao123 class=mnav>hao123</a> <a href=http://map.baidu.com name=tj_trmap class=mnav>地图</a> <a href=http://v.baidu.com name=tj_trvideo class=mnav>视频</a> <a href=http://tieba.baidu.com name=tj_trtieba class=mnav>贴吧</a> <noscript> <a href=http://www.baidu.com/bdorz/login.gif?login&amp;tpl=mn&amp;u=http%3A%2F%2Fwww.baidu.com%2f%3fbdorz_come%3d1 name=tj_login class=lb>登录</a> </noscript> <script>document.write('<a href="http://www.baidu.com/bdorz/login.gif?login&tpl=mn&u='+ encodeURIComponent(window.location.href+ (window.location.search === "" ? "?" : "&")+ "bdorz_come=1")+ '" name="tj_login" class="lb">登录</a>');
                </script> <a href=//www.baidu.com/more/ name=tj_briicon class=bri style="display: block;">更多产品</a> </div> </div> </div> <div id=ftCon> <div id=ftConw> <p id=lh> <a href=http://home.baidu.com>关于百度</a> <a href=http://ir.baidu.com>About Baidu</a> </p> <p id=cp>&copy;2017&nbsp;Baidu&nbsp;<a href=http://www.baidu.com/duty/>使用百度前必读</a>&nbsp; <a href=http://jianyi.baidu.com/ class=cp-feedback>意见反馈</a>&nbsp;京ICP证030173号&nbsp; <img src=//www.baidu.com/img/gs.gif> </p> </div> </div> </div> </body> </html>

-v参数输出通信的整个过程，用于调试。可以看出默认的请求是GET方式

curl -v https://www.baidu.com
* About to connect() to www.baidu.com port 443 (#0)
*   Trying 182.61.200.6...
* Connected to www.baidu.com (182.61.200.6) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* 	subject: CN=baidu.com,O="Beijing Baidu Netcom Science Technology Co., Ltd",OU=service operation department,L=beijing,ST=beijing,C=CN
* 	start date: Apr 02 07:04:58 2020 GMT
* 	expire date: Jul 26 05:31:02 2021 GMT
* 	common name: baidu.com
* 	issuer: CN=GlobalSign Organization Validation CA - SHA256 - G2,O=GlobalSign nv-sa,C=BE
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.baidu.com
> Accept: */*
> 
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
< Connection: keep-alive
< Content-Length: 2443
< Content-Type: text/html
< Date: Mon, 29 Jun 2020 08:22:18 GMT
< Etag: "588603eb-98b"
< Last-Modified: Mon, 23 Jan 2017 13:23:55 GMT
< Pragma: no-cache
< Server: bfe/1.0.8.18
< Set-Cookie: BDORZ=27315; max-age=86400; domain=.baidu.com; path=/
< 
<!DOCTYPE html>
<!--STATUS OK--><html> <head><meta http-equiv=content-type content=text/html;charset=utf-8><meta http-equiv=X-UA-Compatible content=IE=Edge><meta content=always name=referrer><link rel=stylesheet type=text/css href=https://ss1.bdstatic.com/5eN1bjq8AAUYm2zgoY3K/r/www/cache/bdorz/baidu.min.css><title>百度一下，你就知道</title></head> <body link=#0000cc> <div id=wrapper> <div id=head> <div class=head_wrapper> <div class=s_form> <div class=s_form_wrapper> <div id=lg> <img hidefocus=true src=//www.baidu.com/img/bd_logo1.png width=270 height=129> </div> <form id=form name=f action=//www.baidu.com/s class=fm> <input type=hidden name=bdorz_come value=1> <input type=hidden name=ie value=utf-8> <input type=hidden name=f value=8> <input type=hidden name=rsv_bp value=1> <input type=hidden name=rsv_idx value=1> <input type=hidden name=tn value=baidu><span class="bg s_ipt_wr"><input id=kw name=wd class=s_ipt value maxlength=255 autocomplete=off autofocus=autofocus></span><span class="bg s_btn_wr"><input type=submit id=su value=百度一下 class="bg s_btn" autofocus></span> </form> </div> </div> <div id=u1> <a href=http://news.baidu.com name=tj_trnews class=mnav>新闻</a> <a href=https://www.hao123.com name=tj_trhao123 class=mnav>hao123</a> <a href=http://map.baidu.com name=tj_trmap class=mnav>地图</a> <a href=http://v.baidu.com name=tj_trvideo class=mnav>视频</a> <a href=http://tieba.baidu.com name=tj_trtieba class=mnav>贴吧</a> <noscript> <a href=http://www.baidu.com/bdorz/login.gif?login&amp;tpl=mn&amp;u=http%3A%2F%2Fwww.baidu.com%2f%3fbdorz_come%3d1 name=tj_login class=lb>登录</a> </noscript> <script>document.write('<a href="http://www.baidu.com/bdorz/login.gif?login&tpl=mn&u='+ encodeURIComponent(window.location.href+ (window.location.search === "" ? "?" : "&")+ "bdorz_come=1")+ '" name="tj_login" class="lb">登录</a>');
                </script> <a href=//www.baidu.com/more/ name=tj_briicon class=bri style="display: block;">更多产品</a> </div> </div> </div> <div id=ftCon> <div id=ftConw> <p id=lh> <a href=http://home.baidu.com>关于百度</a> <a href=http://ir.baidu.com>About Baidu</a> </p> <p id=cp>&copy;2017&nbsp;Baidu&nbsp;<a href=http://www.baidu.com/duty/>使用百度前必读</a>&nbsp; <a href=http://jianyi.baidu.com/ class=cp-feedback>意见反馈</a>&nbsp;京ICP证030173号&nbsp; <img src=//www.baidu.com/img/gs.gif> </p> </div> </div> </div> </body> </html>
* Connection #0 to host www.baidu.com left intact

--trace参数也可以用于调试，还会输出原始的二进制数据。数据太长只截取了请求部分

curl --trace - https://www.baidu.com
== Info: About to connect() to www.baidu.com port 443 (#0)
== Info:   Trying 182.61.200.7...
== Info: Connected to www.baidu.com (182.61.200.7) port 443 (#0)
== Info: Initializing NSS with certpath: sql:/etc/pki/nssdb
== Info:   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
== Info: SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
== Info: Server certificate:
== Info: 	subject: CN=baidu.com,O="Beijing Baidu Netcom Science Technology Co., Ltd",OU=service operation department,L=beijing,ST=beijing,C=CN
== Info: 	start date: Apr 02 07:04:58 2020 GMT
== Info: 	expire date: Jul 26 05:31:02 2021 GMT
== Info: 	common name: baidu.com
== Info: 	issuer: CN=GlobalSign Organization Validation CA - SHA256 - G2,O=GlobalSign nv-sa,C=BE
=> Send header, 77 bytes (0x4d)
0000: 47 45 54 20 2f 20 48 54 54 50 2f 31 2e 31 0d 0a GET / HTTP/1.1..
0010: 55 73 65 72 2d 41 67 65 6e 74 3a 20 63 75 72 6c User-Agent: curl
0020: 2f 37 2e 32 39 2e 30 0d 0a 48 6f 73 74 3a 20 77 /7.29.0..Host: w
0030: 77 77 2e 62 61 69 64 75 2e 63 6f 6d 0d 0a 41 63 ww.baidu.com..Ac
0040: 63 65 70 74 3a 20 2a 2f 2a 0d 0a 0d 0a          cept: */*....
<= Recv header, 17 bytes (0x11)

-A参数指定客户端的用户代理标头，即User-Agent。curl 的默认用户代理字符串是curl/[version]。（User-Agent它是一个特殊字符串头，是一种向访问网站提供你所使用的浏览器类型及版本、操作系统及版本、浏览器内核、等信息的标识。通过这个标识，用户所访问的网站可以显示不同的排版从而为用户提供更好的体验或者进行信息统计）

//默认是curl的版本
 curl  www.baidu.com -v
* About to connect() to www.baidu.com port 80 (#0)
*   Trying 182.61.200.7...
* Connected to www.baidu.com (182.61.200.7) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.baidu.com
> Accept: */*
>
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
< Connection: keep-alive
< Content-Length: 2381
< Content-Type: text/html
< Date: Tue, 30 Jun 2020 02:55:49 GMT


//设置空
curl -A '' www.baidu.com -v
* About to connect() to www.baidu.com port 80 (#0)
*   Trying 182.61.200.7...
* Connected to www.baidu.com (182.61.200.7) port 80 (#0)
> GET / HTTP/1.1
> Host: www.baidu.com
> Accept: */*
>
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Cache-Control: no-cache
< Connection: keep-alive
< Content-Length: 14615

//设置为ggggg
 curl -A  'ggggg' www.baidu.com -v
* About to connect() to www.baidu.com port 80 (#0)
*   Trying 182.61.200.6...
* Connected to www.baidu.com (182.61.200.6) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: ggggg
> Host: www.baidu.com
> Accept: */*
>
< HTTP/1.1 200 OK
< Bdpagetype: 1
< Bdqid: 0xd8e8994b000077f3
< Cache-Control: private
< Connection: keep-alive
< Content-Type: text/html;charset=utf-8

//也可以通过-H参数设置

 curl -H 'User-Agent: ggggg' www.baidu.com -v
* About to connect() to www.baidu.com port 80 (#0)
*   Trying 182.61.200.7...
* Connected to www.baidu.com (182.61.200.7) port 80 (#0)
> GET / HTTP/1.1
> Host: www.baidu.com
> Accept: */*
> User-Agent: ggggg
>
< HTTP/1.1 200 OK
< Bdpagetype: 1
< Bdqid: 0xd04967af00041e32
< Cache-Control: private
< Connection: keep-alive
< Content-Type: text/html;charset=utf-8

-b参数用来向服务器发送 Cookie。

（由于HTTP是一种无状态的协议，服务器但从网络连接上无从知道用户的身份。怎么办呢？就给客户发一个通行证吧，每人一个，无论谁访问都必须携带自己的通行证。这样服务器就能从通行证上确认客户的身份了。这就是Cookie的工作原理。Cookie实际上是一小段的文本信息。客户端请求服务器，如果服务器需要记录该用户状态，就使用response向客户端浏览器颁发一个Cookie。客户端浏览器会把Cookie保存起来。当浏览器再次请求该网站时，浏览器把请求的网址连同该Cookie一同发送给服务器。服务器检查该Cookie，以此来辨别用户状态。服务器还可以根据需要修改Cookie的内容）

 curl -b "user=root;pass=123456" www.baidu.com -v
或者  curl http://www.baidu.com --cookie "user=root;pass=123456" -v

* About to connect() to www.baidu.com port 80 (#0)
*   Trying 182.61.200.7...
* Connected to www.baidu.com (182.61.200.7) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.baidu.com
> Accept: */*
> Cookie: user=root;pass=123456
>
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
< Connection: keep-alive
< Content-Length: 2381
< Content-Type: text/html
< Date: Tue, 30 Jun 2020 05:53:11 GMT
< Etag: "588604c8-94d"
< Last-Modified: Mon, 23 Jan 2017 13:27:36 GMT
< Pragma: no-cache
< Server: bfe/1.0.8.18
< Set-Cookie: BDORZ=27315; max-age=86400; domain=.baidu.com; path=/


也可以通过读文件

 echo 'user=root;pass=123456' > cookie.txt
[root@test ~]# cat cookie.txt
user=root;pass=123456
[root@test ~]# curl -b cookie.txt   www.baidu.com -v
* About to connect() to www.baidu.com port 80 (#0)
*   Trying 182.61.200.6...
* Connected to www.baidu.com (182.61.200.6) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.baidu.com
> Accept: */*
>
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
< Connection: keep-alive
< Content-Length: 2381
< Content-Type: text/html
< Date: Tue, 30 Jun 2020 06:27:59 GMT
< Etag: "588604c8-94d"
< Last-Modified: Mon, 23 Jan 2017 13:27:36 GMT
< Pragma: no-cache
< Server: bfe/1.0.8.18
* Added cookie BDORZ="27315" for domain baidu.com, path /, expire 1593584879
< Set-Cookie: BDORZ=27315; max-age=86400; domain=.baidu.com; path=/

-c参数将服务器设置的 Cookie 写入一个文件。

 curl -c cookies.txt https://www.baidu.com

--header 有时需要在 http request 之中，自行增加一个头信息。--header 参数就可以起到这个作用

curl --header "Content-Type:application/json" http://www.baidu.com -v
* About to connect() to www.baidu.com port 80 (#0)
*   Trying 182.61.200.6...
* Connected to www.baidu.com (182.61.200.6) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.baidu.com
> Accept: */*
> Content-Type:application/json
>
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
< Connection: keep-alive
< Content-Length: 2381
< Content-Type: text/html
< Date: Tue, 30 Jun 2020 06:43:00 GMT
< Etag: "588604c8-94d"
< Last-Modified: Mon, 23 Jan 2017 13:27:36 GMT
< Pragma: no-cache
< Server: bfe/1.0.8.18
< Set-Cookie: BDORZ=27315; max-age=86400; domain=.baidu.com; path=/

-d参数用于发送 POST 请求的数据体。使用-d参数以后，HTTP 请求会自动加上标头Content-Type : application/x-www-form-urlencoded。并且会自动将请求转为 POST 方法，因此可以省略-X POST。-d参数可以读取本地文本文件的数据，向服务器发送。

 curl -d'login=emma＆password=123'-X POST https://www.baidu.com -v
* Could not resolve host: POST; No address associated with hostname
* Closing connection 0
curl: (6) Could not resolve host: POST; No address associated with hostname
* About to connect() to www.baidu.com port 443 (#1)
*   Trying 182.61.200.7...
* Connected to www.baidu.com (182.61.200.7) port 443 (#1)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=baidu.com,O="Beijing Baidu Netcom Science Technology Co., Ltd",OU=service operation department,L=beijing,ST=beijing,C=CN
*       start date: Apr 02 07:04:58 2020 GMT
*       expire date: Jul 26 05:31:02 2021 GMT
*       common name: baidu.com
*       issuer: CN=GlobalSign Organization Validation CA - SHA256 - G2,O=GlobalSign nv-sa,C=BE
> POST / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.baidu.com
> Accept: */*
> Content-Length: 27
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 27 out of 27 bytes
< HTTP/1.1 302 Found
< Connection: keep-alive
< Content-Length: 17931
< Content-Type: text/html
< Date: Tue, 30 Jun 2020 06:46:51 GMT
< Etag: "54d97487-460b"
< Server: bfe/1.0.8.18

-e参数用来设置 HTTP 的标头Referer，表示请求的来源。-H参数可以通过直接添加标头Referer，达到同样效果

 curl -e 'www.baidu.com' https://www.baidu.com -v
* About to connect() to www.baidu.com port 443 (#0)
*   Trying 182.61.200.7...
* Connected to www.baidu.com (182.61.200.7) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=baidu.com,O="Beijing Baidu Netcom Science Technology Co., Ltd",OU=service operation department,L=beijing,ST=beijing,C=CN
*       start date: Apr 02 07:04:58 2020 GMT
*       expire date: Jul 26 05:31:02 2021 GMT
*       common name: baidu.com
*       issuer: CN=GlobalSign Organization Validation CA - SHA256 - G2,O=GlobalSign nv-sa,C=BE
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.baidu.com
> Accept: */*
> Referer: www.baidu.com
>
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
< Connection: keep-alive
< Content-Length: 2443
< Content-Type: text/html
< Date: Tue, 30 Jun 2020 06:49:01 GMT
< Etag: "588603eb-98b"
< Last-Modified: Mon, 23 Jan 2017 13:23:55 GMT
< Pragma: no-cache
< Server: bfe/1.0.8.18
< Set-Cookie: BDORZ=27315; max-age=86400; domain=.baidu.com; path=/



 curl -H 'Referer:www.baidu.com' https://www.baidu.com -v
* About to connect() to www.baidu.com port 443 (#0)
*   Trying 182.61.200.7...
* Connected to www.baidu.com (182.61.200.7) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=baidu.com,O="Beijing Baidu Netcom Science Technology Co., Ltd",OU=service operation department,L=beijing,ST=beijing,C=CN
*       start date: Apr 02 07:04:58 2020 GMT
*       expire date: Jul 26 05:31:02 2021 GMT
*       common name: baidu.com
*       issuer: CN=GlobalSign Organization Validation CA - SHA256 - G2,O=GlobalSign nv-sa,C=BE
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.baidu.com
> Accept: */*
> Referer:www.baidu.com
>
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
< Connection: keep-alive
< Content-Length: 2443
< Content-Type: text/html
< Date: Tue, 30 Jun 2020 06:50:49 GMT
< Etag: "588603eb-98b"
< Last-Modified: Mon, 23 Jan 2017 13:23:55 GMT
< Pragma: no-cache
< Server: bfe/1.0.8.18
< Set-Cookie: BDORZ=27315; max-age=86400; domain=.baidu.com; path=/
<
<!DOCTYPE html>

-u参数用来设置服务器认证的用户名和密码。

curl -u 'root:12345' https://www.baidu.com  -v
* About to connect() to www.baidu.com port 443 (#0)
*   Trying 182.61.200.6...
* Connected to www.baidu.com (182.61.200.6) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=baidu.com,O="Beijing Baidu Netcom Science Technology Co., Ltd",OU=service operation department,L=beijing,ST=beijing,C=CN
*       start date: Apr 02 07:04:58 2020 GMT
*       expire date: Jul 26 05:31:02 2021 GMT
*       common name: baidu.com
*       issuer: CN=GlobalSign Organization Validation CA - SHA256 - G2,O=GlobalSign nv-sa,C=BE
* Server auth using Basic with user 'root'
> GET / HTTP/1.1
> Authorization: Basic cm9vdDoxMjM0NQ==
> User-Agent: curl/7.29.0
> Host: www.baidu.com
> Accept: */*
>
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
< Connection: keep-alive
< Content-Length: 2443
< Content-Type: text/html
< Date: Tue, 30 Jun 2020 06:53:12 GMT
< Etag: "588603eb-98b"
< Last-Modified: Mon, 23 Jan 2017 13:23:55 GMT
< Pragma: no-cache
< Server: bfe/1.0.8.18
< Set-Cookie: BDORZ=27315; max-age=86400; domain=.baidu.com; path=/
<

以上只是常用参数其他参数可以用curl --help查看

libcurl 使用 https://curl.haxx.se/libcurl/

#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <curl/curl.h>


#define RECV_BUF_LEN 1
char str[RECV_BUF_LEN] = {0};

char *url1 = "www.baidu.com";
char *url2 = "http://blog.sina.com.cn/";
time_t time_start, time_end;

/*size_t write_data(void *ptr, size_t size, size_t nmemb, void *stream)
{
    if (strlen((char *)stream) + strlen((char *)ptr) > RECV_BUF_LEN)
        return 0;
    strcat(stream, (char *)ptr);
    return size * nmemb;
}*/

size_t write_data(void *ptr, size_t size, size_t nmemb, void *stream)
{
    return 0;
}

int curl_easy_demo(int num)
{
    CURL *curl_handle;
    char *curl_data;
    curl_global_init(CURL_GLOBAL_DEFAULT);
    printf("after curl_global_init\n");
    curl_handle = curl_easy_init();
    if (curl_handle == NULL)
    {
        printf("curl_easy_init error!\n");
        return -1;
    }

    curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, write_data); //设置回调函数
    curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, NULL);           //设置回调函数要调用的数据
   // curl_easy_setopt(curl_handle, CURLOPT_CUSTOMREQUEST, "GET");      //设置请求方式
    curl_easy_setopt(curl_handle, CURLOPT_VERBOSE, 1L);                 //打印详细信息
    curl_easy_setopt(curl_handle, CURLOPT_HEADER, 1);                 //显示头部信息
    curl_easy_setopt(curl_handle, CURLOPT_TIMEOUT, 5);                //设置超时时间5秒
    CURLcode res;
    printf("after curl_easy_setopt\n");
        time_start = time(NULL);
    // curl_easy_setopt(curl_handle, CURLOPT_URL, url);
    for (int i = 0; i < num; i++)
    {
        if (i % 2 == 0)
        {
            curl_easy_setopt(curl_handle, CURLOPT_URL, url1);
        }
        else
        {
            curl_easy_setopt(curl_handle, CURLOPT_URL, url2);
        }
        printf("before  curl_easy_perform\n");
        res = curl_easy_perform(curl_handle);
        int resp_code;
        curl_easy_getinfo(curl_handle, CURLINFO_RESPONSE_CODE, &resp_code);
        printf("after  curl_easy_perform %d \n", resp_code);
    }

    time_end = time(NULL);
    curl_easy_cleanup(curl_handle);
    return 0;
}

/**
 * 生成一个easy curl对象，进行一些简单的设置操作
 */
CURL *curl_easy_handler(char *url, char *resp)
{
    CURL *curl = curl_easy_init();

    curl_easy_setopt(curl, CURLOPT_URL, url);
    curl_easy_setopt(curl, CURLOPT_NOSIGNAL, 1);
    //curl_easy_setopt(curl, CURLOPT_HEADER, 1); //显示头部信息
    //curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
    curl_easy_setopt(curl, CURLOPT_TIMEOUT_MS, 10000);

    // write function //
    curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data); //设置回调函数
    curl_easy_setopt(curl, CURLOPT_WRITEDATA, resp);           //设置回调函数要调用的数据

    return curl;
}

/**
 * 使用select函数监听multi curl文件描述符的状态
 * 监听成功返回0，监听失败返回-1
 */
int curl_multi_select(CURLM *curl_m)
{
    int ret = 0;

    struct timeval timeout_tv;
    fd_set fd_read;
    fd_set fd_write;
    fd_set fd_except;
    int max_fd = -1;

    // 注意这里一定要清空fdset,curl_multi_fdset不会执行fdset的清空操作  //
    FD_ZERO(&fd_read);
    FD_ZERO(&fd_write);
    FD_ZERO(&fd_except);

    // 设置select超时时间  //
    timeout_tv.tv_sec = 1;
    timeout_tv.tv_usec = 0;

    // 获取multi curl需要监听的文件描述符集合 fd_set //
    curl_multi_fdset(curl_m, &fd_read, &fd_write, &fd_except, &max_fd);

    /**
     * When max_fd returns with -1,
     * you need to wait a while and then proceed and call curl_multi_perform anyway.
     * How long to wait? I would suggest 100 milliseconds at least,
     * but you may want to test it out in your own particular conditions to find a suitable value.
     */
    if (-1 == max_fd)
    {
        return -1;
    }

    /**
     * 执行监听，当文件描述符状态发生改变的时候返回
     * 返回0，程序调用curl_multi_perform通知curl执行相应操作
     * 返回-1，表示select错误
     * 注意：即使select超时也需要返回0，具体可以去官网看文档说明
     */
    int ret_code = select(max_fd + 1, &fd_read, &fd_write, &fd_except, &timeout_tv);
    switch (ret_code)
    {
    case -1:
        /* select error */
        ret = -1;
        break;
    case 0:
        /* select timeout */
    default:
        /* one or more of curl's file descriptors say there's data to read or write*/
        ret = 0;
        break;
    }

    return ret;
}

typedef struct
{
    char str[32];
} RspArray_s;
RspArray_s RspArray[10000];

int print_out_read(CURLM *curl_m, CURL **CurlArray, int num)
{
    // 输出执行结果 //
    int msgs_left;
    CURLMsg *msg;
    int resp_code;
    /*重复调用这个函数，每次会返回一个新的CURLMsg，直到返回NULL时表明当前已经没有信息可以获取了。msgs_left表示还剩下多少消息*/
    while ((msg = curl_multi_info_read(curl_m, &msgs_left)))
    {
        //printf("msgs_left = %d\n", msgs_left);
        if (CURLMSG_DONE == msg->msg)
        {
            int idx;
            for (idx = 0; idx < num; ++idx)
            {
                if (msg->easy_handle == CurlArray[idx])
                {
                    /*从 curl 句柄里获得附加信息,包括码，Content-Type信息，传输的总耗时(单位为秒)等信息*/
                    curl_easy_getinfo(msg->easy_handle, CURLINFO_RESPONSE_CODE, &resp_code);
                    printf("resp_code = %d\n", resp_code);
                     break;
                }
                  
            }

            if (idx == num)
            {
                //printf("curl not find\n");
            }
            else
            {
                //printf("id = %d completed with status: %d, rep = %s\n", idx, msg->data.result, RspArray[idx].str);
            }
        }
    }
    return 0;
}

int curl_multi_demo_select(int num)
{
    // 初始化一个multi curl 对象 //
    CURLM *curl_m = curl_multi_init();
    //char RspArray[num][32];
    CURL *CurlArray[num];

    // 设置easy curl对象并添加到multi curl对象中  //
    for (int idx = 0; idx < num; ++idx)
    {
        CurlArray[idx] = NULL;
        if (idx % 2 == 0)
        {
            CurlArray[idx] = curl_easy_handler(url1, RspArray[idx].str);
        }
        else
        {
            CurlArray[idx] = curl_easy_handler(url2, RspArray[idx].str);
        }

        if (CurlArray[idx] == NULL)
        {
            return -1;
        }
        curl_multi_add_handle(curl_m, CurlArray[idx]);
    }
    time_start = time(NULL);
    /*
     * 调用curl_multi_perform函数执行curl请求
     * url_multi_perform返回CURLM_CALL_MULTI_PERFORM时，表示需要继续调用该函数直到返回值不是CURLM_CALL_MULTI_PERFORM为止
     * running_handles变量返回正在处理的easy curl数量，running_handles为0表示当前没有正在执行的curl请求
     */
    int running_handles;
    while (CURLM_CALL_MULTI_PERFORM == curl_multi_perform(curl_m, &running_handles))
    {
        printf("running_handles1 = %d\n", running_handles);
    }
    printf("running_handles2 = %d\n", running_handles);
    /**
     * 为了避免循环调用curl_multi_perform产生的cpu持续占用的问题，采用select来监听文件描述符
     */
    while (running_handles)
    {
        if (-1 == curl_multi_select(curl_m))
        {
            printf("select error \n");
            break;
        }
        else
        {
            // select监听到事件，调用curl_multi_perform通知curl执行相应的操作
            while (CURLM_CALL_MULTI_PERFORM == curl_multi_perform(curl_m, &running_handles))
            {
                //printf("select1 running_handles = %d\n", running_handles);
            }
        }
        printf("select2 running_handles = %d\n", running_handles);
    }

    //print_out_read(curl_m, CurlArray, num);
    int resp_code;
     for (int i = 0; i < num; i++)
    {
        curl_easy_getinfo(CurlArray[i], CURLINFO_RESPONSE_CODE, &resp_code);
        printf("resp_code = %d\n", resp_code);
    }
    time_end = time(NULL);
    // 这里要注意cleanup的顺序 //
    for (int idx = 0; idx < num; ++idx)
    {
        curl_multi_remove_handle(curl_m, CurlArray[idx]);
    }

    for (int idx = 0; idx < num; ++idx)
    {
        curl_easy_cleanup(CurlArray[idx]);
    }

    curl_multi_cleanup(curl_m);

    return 0;
}

int curl_multi_demo_wait(int num)
{
    // 初始化一个multi curl 对象 //
    CURLM *curl_m = curl_multi_init();
    //char RspArray[num][32];
    CURL *CurlArray[num];

    // 设置easy curl对象并添加到multi curl对象中  //
    for (int idx = 0; idx < num; ++idx)
    {
        CurlArray[idx] = NULL;
        if (idx % 2 == 0)
        {
            CurlArray[idx] = curl_easy_handler(url1, RspArray[idx].str);
        }
        else
        {
            CurlArray[idx] = curl_easy_handler(url2, RspArray[idx].str);
        }

        if (CurlArray[idx] == NULL)
        {
            return -1;
        }
        curl_multi_add_handle(curl_m, CurlArray[idx]);
    }
    time_start = time(NULL);
    /*
     * 调用curl_multi_perform函数执行curl请求
     * url_multi_perform返回CURLM_CALL_MULTI_PERFORM时，表示需要继续调用该函数直到返回值不是CURLM_CALL_MULTI_PERFORM为止
     * running_handles变量返回正在处理的easy curl数量，running_handles为0表示当前没有正在执行的curl请求
     */
    int still_running;
    //发送数据并用still_running监测数据是否发送完成，如果未发送完成则持续发送

    curl_multi_perform(curl_m, &still_running);
    //printf("after curl_multi_perform still_running = %d\n", still_running);
    do
    {
        int numfds = 0;
        int res = curl_multi_wait(curl_m, NULL, 0, 30000, &numfds);
        if (res != CURLM_OK)
        {
            printf("error: curl_multi_wait() returned %d\n", res);
            return -1;
        }
        curl_multi_perform(curl_m, &still_running);
        //printf("curl_multi_perform still_running = %d\n", still_running);
    } while (still_running);

    print_out_read(curl_m, CurlArray, num);
    time_end = time(NULL);
    // 这里要注意cleanup的顺序 //
    for (int idx = 0; idx < num; ++idx)
    {
        curl_multi_remove_handle(curl_m, CurlArray[idx]);
    }

    for (int idx = 0; idx < num; ++idx)
    {
        curl_easy_cleanup(CurlArray[idx]);
    }

    curl_multi_cleanup(curl_m);
}

int main(int argc, char **argv)
{

    if (argc < 3)
    {
        printf("input request num \n");
        return 0;
    }
    int opt = atoi(argv[1]);
    int num = atoi(argv[2]);
    if (opt == 1)
    {
        printf("curl_easy_demo\n");
        curl_easy_demo(num);
    }
    else if (opt == 2)
    {
        printf("curl_multi_demo_select\n");
        curl_multi_demo_select(num);
    }
    else if (opt == 3)
    {
        printf("curl_multi_demo_wait\n");
        for(int i = 0; i < 10; i++)
        {
            printf("**********************\n");
            curl_multi_demo_wait(num);
        }
    }
    else
    {
        return 0;
    }
    printf("start = %ld, end = %ld\n", time_start, time_end);
    return 0;
}

耿小渣

关注

0
点赞
踩
1

收藏

觉得还不错? 一键收藏
0
评论
curl使用

curl 是常用的命令行工具，用来请求 Web 服务器。它的名字就是客户端（client）的 URL 工具的意思。curl https://www.baidu.com<!DOCTYPE html><html> <head><meta http-equiv=content-type content=text/html;charset=utf-8><meta http-equiv=X-UA-Compat
复制链接

扫一扫

专栏目录