urllib库应用简单整合

 这些是我对urllib库的使用的简单整合，我把这些东西都整合在一个文档里面，为的是方便我在使用时查看。没有官方网站的详细，但是这个我觉得对于我这种习惯短暂性遗忘的人来说有点用处，分享出来，希望也能帮到你们^_^

Urllib库
Urllib.request.urlopen(url, data=None, [timeout, ]*,cafile=None,cadefault=False, context=None)
Import urllib.request
Response=urllib.request.urlopen(‘http://...’)
Print(response.read().decode(‘utf-8’))   get请求
Import urllib.request
Import urllib.parse
data=bytes(urllib.parse.urlencode({‘word’:’hello’}),encoding=’utf-8’)
Response=urllib.request.urlopen(‘http://httpbin.org/post’,data=data)
Print(response.read())  post请求
Import urllib.request
Response=urllib.request.urlopen(‘http://.....’,timeout=1)
Print(response.read())  设置时间
import socket
Import urllib.request
Import urllib.error
Try:
Response=urllib.request.urlopen(‘http://....’,timeout=1)
Except urllib.error.URLError as e:
If isinstance(e.reason,socket.timeout):
Print(‘time out’)  错误设置
状态码，响应头
Import urllib.request
Response=urllib.request.urlopen(‘http://...’)
Print(response.status)响应状态
Print(response.getheaders())获取响应头
Print(response.getheader(‘Server’))获取特定的响应头server
Request用法
Import urllib.request
request=urllib.request.Request(‘http’)将url转化为对象
Response=urllib.request.urlopen(request)
Print(response..read().decode(‘utf-8’))
详细
（1）from urllib import request,parse
url=’http://httpbin.org/post’
headers={
‘User-Agent’:” ”
‘Host’:” ”
}
dict={
‘name’:’something’
}
data=bytes(parse.urlencode(dict),encoding=’utf-8’)
req=request.Request(url=url,data=data,headers=headers,method=’POST’)结果包含所有信息
response=request.urlopen(req)
print(response.read().decode(‘utf-8’))
（2）from urllib import request, parse
url=”...”
dict={
‘name’:” ”}
data =bytes(parse.urlencode(dict),encoding=’utf-8’)
req =request.Request(url=url,data=data,method=’POST’)
req.add_header(“header内容”)
Response=request.urlopen(req)
Print(response.read().decode(‘utf-8’))
Handler代理
Import urllib.request
proxy_handler=Cliburn.request.ProxyHandler(
{‘http’:’http://ip地址’或者
‘https’:’https://ip’})
opener=urllib.request.build_opener(proxy_handler)
response=opener.open(‘url’)  print(response.read())
Cookie 使用
(1)import http.cookiejar,urllib.request
cookie=http.cookiejar.Cookiejar()
handler=urllib.request.HTTPCookieProcessor(cookie)
opener=urllib.request.builder_opener(handler)
response=opener.open(‘url’)
for item in cookie:
print(item.name+’=’+item.value
(2)import http.cookiejar,urllib.request
filename=’cookie.text’
cookie=http.cookiejar.MozillaCookieJar(filename)
handler=urllib.request.HTTPCookieProcessor(cookie)
opener=urllib.request.build_opener(handler)
response=opener.open(‘url’)
cookie.save(ignore_discard=True,ignore_expires=True)
异常处理
(1)from urllib import request,error
try:
response=request.urlopen(‘url’)
except error.URLError as e:
print(e.reason)
(2)from urllib import request,error
try：
response=request.urlopen(‘url’)
except error.HTTPError as e:
print(e.reason,e.code,e.headers,sep=’\n’)
except error.URLError as e:
print(e.reason)
else:
print(‘Request Successfuly’)
(3)import socket
import urllib.request
import urllib.error
try:
response=urllib.request.urlopen(‘url’,timeout=1)
except urllib.error.URLError as e:
print(e.reason)
if isinstance(e.reason,socket.timeout)
print(‘time out’)
url解析
urllib.parse.urlparse(urlstring,scheme=’ ‘,allow_fregments=True)
(1)from urllib.parse import urlparse
result=urlparse(‘url’)
print(type(result),result)
(2)from urllib.parse import urlparse
result=urlparse(‘url’,scheme=’https’/allow_fragments=False)  print(result)
urlunparse 将url进行拼接
from urllib.parse import urlunparse
data=[‘http’,’www.baidu.com’,’index.html’,’user’,’a=6’,’comment’]
print(unparse(data))   http://www.baidu.com/index.html;user?a=6#comment
urljoin url拼接 前后拼接，但是后面的会覆盖前面的字段
urlencode 将字典格式转化为get请求参数
from urllib.parse import  urlencode
params={
‘name’:’germey’
‘age’:22}
base_url=’http://www.baidu.com?’
url=base_url+urlencode(params)  print(url)
http://www.baidu.com?name=germey&age=22