使用python3模拟http头部信息

有的网站为了防止恶意爬取数据，对UA头进行了校验，判断是否是来自于浏览器的请求，我们使用python3请求的UA头是 “User-Agent”: “Python-urllib/3.5”，只要将UA头指定为浏览器UA头或者任意内容。

root@VM-0-7-ubuntu:~/python/zeropython3/15# cat post_4.py 
from urllib import parse
from urllib import request

headers={
"User-Agent":"I am UA!"
}

data = bytes(parse.urlencode({'hello':'world'}),encoding = 'utf8')

req = request.Request('http://httpbin.org/post',data=data,headers=headers,method = 'POST')
response = request.urlopen(req)
print (response.read().decode('utf-8'))
------------------------------------------------------------------------------------------
root@VM-0-7-ubuntu:~/python/zeropython3/15# python3 post_4.py 
{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {
    "hello": "world"
  }, 
  "headers": {
    "Accept-Encoding": "identity", 
    "Content-Length": "11", 
    "Content-Type": "application/x-www-form-urlencoded", 
    "Host": "httpbin.org", 
    "User-Agent": "I am UA!"  #成功替换
  }, 
  "json": null, 
  "origin": "xxxxx, xxxxxxx", 
  "url": "https://httpbin.org/post"
}