Requests库详解_requests l-CSDN博客

本文链接：https://blog.csdn.net/l4642247/article/details/81017295

什么是Urllib

内置的一个http请求库，不需要额外的安装，不需要了解底层到底怎么实现。

urllib.request 请求模块
urllib.error 异常处理模块
urllib.parse url解析模块
urllib.robotparser

import urllib2
response = urllib2.urlopen('http://www.baidu.com')

什么是Requests

基于urllib3，Python实现的简单易用的http请求库

相关用法

基本GET请求

import requests

response = requests.get('http://httpbin.org/get')
print response.text

请求结果：

{
	"args": {},
	"headers": {
		"Accept": "*/*",
		"Accept-Encoding": "gzip, deflate",
		"Connection": "close",
		"Host": "httpbin.org",
		"User-Agent": "python-requests/2.19.1"
	},
	"origin": "223.104.213.74",
	"url": "http://httpbin.org/get"
}

带参数的GET请求，字典方式传值

import requests

data = {
        'name': 'lt',
        'age': 18
    }

response = requests.get('http://httpbin.org/get', params=data)
print response.text

请求结果：

{
	"args": {
		"age": "18",
		"name": "lt"
	},
	"headers": {
		"Accept": "*/*",
		"Accept-Encoding": "gzip, deflate",
		"Connection": "close",
		"Host": "httpbin.org",
		"User-Agent": "python-requests/2.19.1"
	},
	"origin": "223.104.213.74",
	"url": "http://httpbin.org/get?age=18&name=lt"
}

解析Json

import requests

data = {
        'name': 'lt',
        'age': 18
    }

response = requests.get('http://httpbin.org/get', params=data)
print response.json()

获取二进制数据

import requests

response = requests.get('https://ss1.bdstatic.com/kvoZeXSm1A5BphGlnYG/skin_zoom/178.jpg?2')
with open('e:/aaa.jpg', 'wb') as f:
    f.write(response.content)
    f.close()

添加一个headers，伪装

不加headers，返回400：

import requests

response = requests.get('https://www.zhihu.com/')
print response.status_code

加了之后，返回200：

import requests

headers = {
    'user-agent' : 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36'
    }
response = requests.get('https://www.zhihu.com/', headers = headers)
print response.status_code

POST请求

import requests

data = {
    'aaaa' : 'bbbb'
    }
response = requests.post('http://httpbin.org/post', data = data)
print response.json()

响应

response属性

status_code
headers
cookies
url
history

状态码的判断

import requests

response = requests.post('http://httpbin.org/post')
print response.status_code == 200

高级操作

文件上传

import requests

files = {'file': open('e:/aaa.jpg', 'rb')}
response = requests.post('http://httpbin.org/post',files = files)
print response.json()

获取cookies

import requests

response = requests.get('http://www.baidu.com')
print response.cookies
for key,value in response.cookies.items():
    print(key + ' = ' + value)

会话维持（用作登录验证）

如果是：

import requests

requests.get('http://httpbin.org/cookies/set/number/123456')
response = requests.get('http://httpbin.org/cookies')
print response.text

{"cookies":{}}

改为：

import requests

s = requests.session()
s.get('http://httpbin.org/cookies/set/number/123456')
response = s.get('http://httpbin.org/cookies')
print response.text

{"cookies":{"number":"123456"}}

证书验证

证书是不合法的，这种情况下会报

requests.exceptions.SSLError

import requests

response = requests.get('https://www.12306.cn')
print response.status_code

改为

import requests
import urllib3

urllib3.disable_warnings() #消除警告
response = requests.get('https://www.12306.cn', verify=False)
print response.status_code

代理设置

import requests

proxies = {
    'http':'http://127.0.0.1:8743',
    'https':'https://127.0.0.1:9743'
    }
response = requests.get('https://www.taobao.com', proxies = proxies)
print response.status_code

超时的设置

import requests

try:
    requests.get('https://www.taobao.com/', timeout=0.1)
except requests.exceptions.ConnectTimeout:
    print 'ConnectTimeout'
except requests.exceptions.Timeout:
    print 'Timeout'