欢迎进行接口(httprunner)、UI自动化(pytest)交流,博主微信:jiaotengfei1016
Requests 的高级特性。
一、会话对象session
Session 对象允许您跨请求保留某些参数。向同一主机发出多个请求,底层 TCP 连接将被重用,import requests
# (1)请求中保留cookie:
s = requests.Session()
s.get('https://httpbin.org/cookies/set/sessioncookie/123456789')
r = s.get('https://httpbin.org/cookies')
print(r.text)
# '{"cookies": {"sessioncookie": "123456789"}}'
#(2)向请求方法提供默认数据
s = requests.Session()
s.auth = ('user', 'pass')
s.headers.update({'x-test': 'true'})
# 这里的headers值会被替换,方法级参数覆盖会话参数
s.get('https://httpbin.org/headers', headers={'x-test2': 'true'})
#(3)方法级参数也不会跨请求持久化,只会,这个例子只会发送第一个请求的 cookie,而不是第二个:
s = requests.Session()
r = s.get('https://httpbin.org/cookies', cookies={'from-my': 'browser'})
print(r.text) # '{"cookies": {"from-my": "browser"}}'
r = s.get('https://httpbin.org/cookies')
print(r.text) # '{"cookies": {}}'
# 将cookies添加到会话中,退出with则会话关闭,示例:
with requests.Session() as s:
r = s.get('https://httpbin.org/cookies/set/sessioncookie/123456789')
print(r.text) # '{"cookies": {"from-my": "browser"}}'
r = s.get('https://httpbin.org/cookies')
print(r.text) # '{"cookies": {"from-my": "browser"}}'
二、请求和响应对象
代码如下(示例):
import requests
r = requests.get('https://en.wikipedia.org/wiki/Monty_Python')
print(r.headers) # 返回信息的headers信息
print(r.request.headers) # 请求中的headers信息
三、SSL证书
Requests 默认附带了一套它信任的根证书,Requests可以为HTTPS请求验证SSL证书,就像web浏览器一样。SSL验证默认是开启的,如果证书验证失败,Requests 会抛出 SSLError:
“”":
import requests
r = requests.get('https://requestb.in') # 报错:requests.exceptions.SSLError: hostname 'requestb.in' doesn't match either of '*.herokuapp.com', 'herokuapp.com'
r = requests.get('https://github.com', verify=False) # <Response [200]>
# (6)对响应体进行判断
with requests.get('http://httpbin.org/get', stream=True) as r:
if int(r.headers['content-length']) == 200:
content = r.content
...
四、对响应体进行判断
对响应体进行判断:考虑使用 with 语句发送请求,这样可以保证请求一定会被关闭。代码如下(示例):
import requests
with requests.get('http://httpbin.org/get', stream=True) as r:
if int(r.headers['content-length']) == 200:
content = r.content
...
五、文件过大,使用流式上传模式
要流式传输和上传,只需为您的身体提供一个类似文件的对象::
import requests
with open('massive-body', 'rb') as f:
requests.post('http://some.url/streamed', data=f)
六、一次性上传多个文件
在一个请求中发送多个文件。例如,假设您要将图像文件上传到具有多个文件字段“图像”的 HTML 表单:
<input type="file" name="images" multiple="true" required="true"/>
url = 'https://httpbin.org/post'
files = [
('images', ('foo.png', open('foo.png', 'rb'), 'image/png')),
('images', ('bar.png', open('bar.png', 'rb'), 'image/png'))]
r = requests.post(url, files=files)
r.text
{
...
'files': {'images': 'data:image/png;base64,iVBORw ....'}
'Content-Type': 'multipart/form-data; boundary=3131623adb2043caaeb5538cc7aa0b3a',
...
}
七、事件钩子
Requests 有一个钩子系统,您可以使用它来操纵请求过程的某些部分或信号事件处理,可以通过将字典传递给请求参数来为每个请求分配一个钩子函数 :{hook_name: callback_function}hooks,可用挂钩::
import requests
def record_hook(r, *args, **kwargs):
r.hook_called = True
return r
def print_url(r, *args, **kwargs):
print(r.url)
(1)添加1个钩子函数
requests.get('https://httpbin.org/', hooks={'response': print_url})
返回:
https://httpbin.org/
<Response [200]>
(2)添加2个钩子函数
r = requests.get('https://httpbin.org/', hooks={'response': [print_url, record_hook]})
r.hook_called
(3)向Session实例添加钩子。您添加的任何钩子都将在对会话发出的每个请求中被调用。例如:
s = requests.Session()
s.hooks['response'].append(print_url)
s.get('https://httpbin.org/')
返回:
https://httpbin.org/
<Response [200]>
八、流式请求
Response.iter_lines()迭代数据流:
import json
import requests
r = requests.get('https://httpbin.org/stream/20', stream=True)
for line in r.iter_lines():
if line:
decoded_line = line.decode('utf-8')
print(json.loads(decoded_line))
九、代理
(1)为单个请求配置代理:
import requests
proxies = {
'http': 'http://10.10.1.10:3128',
'https': 'http://10.10.1.10:1080',
}
requests.get('http://example.org', proxies=proxies)
(2)为多个请求配置代理:
import requests
proxies = {
'http': 'http://10.10.1.10:3128',
'https': 'http://10.10.1.10:1080',
}
s = requests.Session()
s.proxies.update(proxies)
s.get('http://example.org')
(3)HTTP 身份验证与代理一起用:
import requests
proxies = {'http': 'http://user:pass@10.10.1.10:3128/'}
(4)为特定主机提供代理:
import requests
proxies = {'http://10.20.1.128': 'http://10.10.1.10:5323'}
(5)为 https 连接使用代理通常需要您的本地计算机信任代理的根证书:
from requests.utils import DEFAULT_CA_BUNDLE_PATH
print(DEFAULT_CA_BUNDLE_PATH)
(6)除了基本的 HTTP 代理之外,Requests 还支持使用 SOCKS 协议的代理pip install requests[socks]:
import requests
proxies = {
'http': 'socks5://user:pass@host:port',
'https': 'socks5://user:pass@host:port'
}
示例代码:
proxies={'http':'socks5://127.0.0.1:1080','https':'socks5:/127.0.0.1.1080'}
url='https://www.facebook.com'
#下面这样访问是会报错,因为没有用代理服务器,直接访问有防火墙
r1=requests.get(url,timeout=10) # 没设置代理
r2=requests.get(url,proxies=proxies,timeout=10) # 设置代理
print(r1.status_code)
print(r2.status_code)
十、其他
(1)r.json()提取返回的信息:
import requests
r = requests.get('https://api.github.com/repos/psf/requests/git/commits/a050faf084662f3a352dd1a941f2c7c9f886d4ad')
# 查看返回的类型
print(r.headers['content-type']) # application/json; charset=utf-8
commit_data = r.json()
# 获取json格式的相关信息
print(commit_data['message'])
print(commit_data['status_code'])
print(commit_data[0].keys()) # ['body', 'url', 'created_at', 'updated_at', 'user', 'id']
print(commit_data[2]['body']) # Probably in the "advanced" section
(2)一种非常基本的身份验证:
from requests.auth import HTTPBasicAuth
auth = HTTPBasicAuth('fake@example.com', 'not_a_real_password')
r = requests.post(url=url, data=body, auth=auth)
r.status_code # 201
content = r.json()
print(content['body']) # Sounds great! I'll get right on it.