055/056_爬虫_urllib_下载/请求对象的定制

下载案例

import urllib.request
# 网页下载
url_page = "http://www.baidu.com"
urllib.request.urlretrieve(url_page,"baidu.html")

# 图片下载
url_img = "https://pic2.zhimg.com/v2-a4619293e8e376f5ee1fc1dd616b4197_r.jpg?source=1940ef5c"
urllib.request.urlretrieve(url_img,"duolaameng.jpg")

# 视频下载
url_video = "https://vd2.bdstatic.com/mda-ikvdq36633tc3bwu/sc/mda-ikvdq36633tc3bwu.mp4?v_from_s=hkapp-haokan-hnb&auth_key=1652528717-0-0-8ae606a3eb512f1a5b1216e2cde9709b&bcevod_channel=searchbox_feed&pd=1&cd=0&pt=3&logid=0916865844&vid=15764645099659448706&abtest=101830_2-102133_1-102148_1-17451_2&klogid=0916865844"
urllib.request.urlretrieve(url_video,"yanhua.mp4")

---

请求对象的定制

url的组成

https://www.baidu.com/s?wd=周杰伦

http/https	www.baidu.com	80/443	s	wd=周杰伦	#
协议	主机	端口号	路径	参数	锚点

端口号

http	https	mysql	Oracle	redis	mongodb
80	443	3306	1521	6379	27017

import urllib.request
url = "http://www.baidu.com"
headers = {
    "User-Agent": "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; SE 2.X MetaSr 1.0; SE 2.X MetaSr 1.0; .NET CLR 2.0.50727; SE 2.X MetaSr 1.0)"
}
# 因为urlopen方法中不能储存字典 所有hraders不能传递进去
# 请求对象的定制
# 注意 因为参数顺序的问题  不能直接写url和headers    中间还有个data 所有我们需要关键字传递
request = urllib.request.Request(url=url,headers=headers)
response = urllib.request.urlopen(request)
content = response.read().decode("utf-8")
print(content)