Python爬虫--隐藏

python爬虫笔记–隐藏

隐藏

方法一：
    #通过Request的header参数修改
    #head={}
    #head["User-Agent"]='User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36'
   		 方法二
	req=urllib.request.Request(url,data)
    #通过Request.add_header()方法修改
    req.add_header('User-Agent','Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36')

代理

1参数是一个字典{“类型”：‘代理ip：端口号’}
proxy_support=urllib.request.ProxyHandler({})
2订制、创建一个oppener
opener=urllib.request.build_opener(proxy_support)
3.1安装opener
urllib.request.install_opener(opener)
3.2调用opener
opener.open(url)

例1

  import urllib.request
url="http://www.whatismyip.com.tw"
proxy_support=urllib.request.ProxyHandler({'http':'180.122.180.192:9999'})
opener = urllib.request.build_opener(proxy_support)
urllib.request.install_opener(opener)

response=urllib.request.urlopen(url)
html=response.read().decode('utf-8')
print(html)