一.基本用法
-
请求类型
r = requests.post('http://httpbin.org/post') r = requests.put('http://httpbin.org/put') r = requests.delete('http://httpbin.org/delete') r = requests.head('http://httpbin.org/get') r = requests.options('http://httpbin.org/get')
-
GET请求
-
参数
-
params
import requests data = { 'name': 'germey', 'age': 22 } r = requests.get("http://httpbin.org/get", params=data) print(r.text)
-
headers
headers = { 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36' } r = requests.get("https://www.zhihu.com/explore", headers=headers)
-
-
json格式
import requests r = requests.get("http://httpbin.org/get") print(type(r.text)) print(r.json()) print(type(r.json()))
-
抓取二进制数据
import requests r = requests.get("https://github.com/favicon.ico") with open('favicon.ico', 'wb') as f: f.write(r.content)
-
-
POST请求
import requests data = {'name': 'germey', 'age': '22'} r = requests.post("http://httpbin.org/post", data=data) print(r.text)
-
响应
发送请求后,得到的自然就是响应。在上面的实例中,我们使用text和content获取了响应的内容。此外,还有很多属性和方法可以用来获取其他信息,比如状态码、响应头、Cookies等
import requests r = requests.get('http://www.jianshu.com') print(type(r.status_code), r.status_code) print(type(r.headers), r.headers) print(type(r.cookies), r.cookies) print(type(r.url), r.url) print(type(r.history), r.history)
二.高级用法
-
文件上传
import requests files = {'file': open('favicon.ico', 'rb')} r = requests.post("http://httpbin.org/post", files=files) print(r.text)
-
cookies
获取Cookies
import requests r = requests.get("https://www.baidu.com") print(r.cookies) for key, value in r.cookies.items(): print(key + '=' + value)
<RequestsCookieJar[<Cookie BDORZ=27315 for .baidu.com/>, <Cookie __bsi=13533594356813414194_00_14_N_N_2_0303_C02F_N_N_N_0 for .www.baidu.com/>]> BDORZ=27315 __bsi=13533594356813414194_00_14_N_N_2_0303_C02F_N_N_N_0
cookies参数来设置
不过这样就需要构造RequestsCookieJar对象,而且需要分割一下cookies
import requests cookies = '...' jar = requests.cookies.RequestsCookieJar() headers = { 'Host': 'www.zhihu.com', 'User-Agent': '...' } for cookie in cookies.split(';'): key, value = cookie.split('=', 1) jar.set(key, value) r = requests.get("http://www.zhihu.com", cookies=jar, headers=headers) print(r.text)
-
会话维持
import requests s = requests.Session() s.get('http://httpbin.org/cookies/set/number/123456789') r = s.get('http://httpbin.org/cookies') print(r.text)
-
SSL证书验证 requests还提供了证书验证的功能。当发送HTTP请求的时候,它会检查SSL证书,我们可以使用verify参数控制是否检查此证书。其实如果不加verify参数的话,默认是True,会自动验证
import requests response = requests.get('https://www.12306.cn') print(response.status_code)
-
代理设置
import requests proxies = { "http": "http://10.10.1.10:3128", "https": "http://10.10.1.10:1080", } requests.get("https://www.taobao.com", proxies=proxies)
HTTP Basic Auth
import requests proxies = { "http": "http://user:password@10.10.1.10:3128/", } requests.get("https://www.taobao.com", proxies=proxies)
SOCKS协议代理
import requests proxies = { 'http': 'socks5://user:password@host:port', 'https': 'socks5://user:password@host:port' } requests.get("https://www.taobao.com", proxies=proxies)
-
超时设置
import requests r = requests.get("https://www.taobao.com", timeout = 1) print(r.status_code)
-
身份认证
import requests from requests.auth import HTTPBasicAuth r = requests.get('http://localhost:5000', auth=HTTPBasicAuth('username', 'password')) print(r.status_code)
简写
import requests r = requests.get('http://localhost:5000', auth=('username', 'password')) print(r.status_code)
OAuth认证
pip3 install requests_oauthlib
import requests from requests_oauthlib import OAuth1 url = 'https://api.twitter.com/1.1/account/verify_credentials.json' auth = OAuth1('YOUR_APP_KEY', 'YOUR_APP_SECRET', 'USER_OAUTH_TOKEN', 'USER_OAUTH_TOKEN_SECRET') requests.get(url, auth=auth)
-
Prepared Request 前面介绍urllib时,我们可以将请求表示为数据结构,其中各个参数都可以通过一个Request对象来表示。这在requests里同样可以做到,这个数据结构就叫Prepared Request。
from requests import Request, Session url = 'http://httpbin.org/post' data = { 'name': 'germey' } headers = { 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36' } s = Session() req = Request('POST', url, data=data, headers=headers) prepped = s.prepare_request(req) r = s.send(prepped) print(r.text)