在进行爬虫的时候,可能会需要添加代理
from urllib.error import URLError
from urllib.request import ProxyHandler,build_opener
proxy_handler=ProxyHandler({
'http':'http://127.0.0.1.8041',
'https':'https://127.0.0.1.8041',
})
opener=build_opener(proxy_handler)
try:
response=opener.open('https://www.baidu.com')
print(response.read().decode('utf-8'))
except URLError as e:
print(e.reason)
在本地搭建搭建一个代理,使之运行在8041的端口上