3-1 urllib基本用法

最新推荐文章于 2024-07-14 20:20:34 发布

huanglianggu

最新推荐文章于 2024-07-14 20:20:34 发布

阅读量141

点赞数

分类专栏： python遇见数据采集

本文链接：https://blog.csdn.net/huanglianggu/article/details/114925565

版权

python遇见数据采集专栏收录该内容

2 篇文章 0 订阅

订阅专栏

3-1 urllib基本用法

C:\Users\Administrator\PycharmProjects\python_data_collection\python_data_collection\urllib_hlg_test.py

from urllib import request

resp = request.urlopen("http://www.baidu.com")
print(resp.read().decode("utf-8"))

C:\Users\Administrator\PycharmProjects\python_data_collection\python_data_collection\urllib_hlg_test.py

from urllib import request

req = request.Request("http://www.baidu.com")
req.add_header("User-Agent", "Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Mobile Safari/537.36")
resp = request.urlopen(req)
print(resp.read().decode("utf-8"))

https://www.thsrc.com.tw/ArticleContent/a3b630bb-1066-4352-a1ef-58c7b4e8ef7c

from urllib.request import urlopen
from urllib.request import Request
from urllib import parse

req = Request("https://kyfw.12306.cn/otn/leftTicket/init")
postData = parse.urlencode([
    ("leftTicketDTO.train_date","2021-03-18"),
("leftTicketDTO.from_station","SHH"),
("leftTicketDTO.to_station","BZH"),
("purpose_codes","ADULT"),
])

req.add_header("Referrer Policy","strict-origin-when-cross-origin")
req.add_header("User-Agent","Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Mobile Safari/537.36")

resp = urlopen(req, data=postData.encode("utf-8"))

print(resp.read().decode("utf-8"))