request模块
1.Python的标准库中 urllib 模块已经包含了平常我们使用的大多数功能,但是它的 API 使用起来让人感觉不太好,Requests 继承了urllib的所有特性,并且API使用更加方便,可以简化我们的代码
get请求
url = “https://xueqiu.com/v4/statuses/public_timeline_by_category.json?”
parmas = {
‘since_id’:’-1’,
‘max_id’:’-1’,
‘count’:‘10’,
‘category’:‘6’,
‘123’:‘中午问’
}
#请求头
headers = {
‘User-Agent’:‘Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36’,
}
response = requests.get(url=url,params=parmas,headers=headers)
响应结果处理
- response.text 返回解码后的字符串
-
- respones.content 以字节形式(二进制)返回。
-
- response.status_code 响应状态码
-
- response.request.headers 请求的请求头
-
- response.headers 响应头
-
- response.encoding = ‘utf-8’ 可以设置编码类型
-
- response.encoding 获取当前的编码
-
- response.json() 内置的JSON解码器,以json形式返回,前提返回的内容确保是json格式的,不然解析出错会抛异常
post请求
需要提交的表单数据
formdata = {
‘name’: ‘18518753265’,
‘password’: ‘ljh123456’,
‘remem_pass’: ‘on’,
‘_s_x_id’: ‘1e56aee116b168e393cf4591b48c44a2’,
‘ljg_login’: ‘1’,
‘m_p_l’: ‘1’,
‘channel’:’’,
‘position’:’’,
}
url:要请求的url地址
data:要提交的表单数据(post请求需要提交的数据)
headers = {
‘User-Agent’:‘Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36’
}
response = requests.post(url=url,data=formdata,headers=headers)
print(response.status_code)
文件上传
import requests
#文件上传
url = ‘https://httpbin.org/post’
files = {‘file’:open(‘page.html’,‘r’)}
headers = {
‘User-Agent’:‘Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36’
}
response=requests.post(url=url,files=files,headers=headers)
if response.status_code == 200:
print(response.text)
web客户验证
import requests
auth=(‘test’, ‘123456’)
response = requests.get(
‘http://192.168.199.107’,
auth = auth
)
print (response.text)
设置代理
proxy = {
‘https’: ‘60.190.250.120:8080’,
‘http’: ‘121.61.3.209:9999’,
}
response=requests.get(url=url,headers=headers,proxies=proxy)
session状态保持
创建session对象,可以保存Cookie值
ssion = requests.session()
处理 headers
headers = {
“User-Agent”: “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36”
}
需要登录的用户名和密码
data = {
“email”:“18518753265”,
“password”:“ljh123456”
}
发送附带用户名和密码的请求,并获取登录后的Cookie值,保存在ssion里
ssion.post(
“http://www.renren.com/PLogin.do”,
data = data
)
ssion包含用户登录后的Cookie值,可以直接访问那些登录后才可以访问的页面
response = ssion.get(
“http://www.renren.com/965722397/profile”
)
打印响应内容
print (response.text)
处理https请求 忽略ssl证书验证
response = requests.get(url=url,headers=headers,verify=False)