urllib的粗略尝试-CSDN博客

本文链接：https://blog.csdn.net/weixin_52634719/article/details/119818745

这篇博客展示了如何使用Python的urllib库进行GET和POST请求，包括设置超时、添加请求头以及处理响应。示例代码中包含了对URL的GET和POST操作，以及如何处理可能出现的超时错误。同时，还演示了如何构造带有特定User-Agent的POST请求。

摘要由CSDN通过智能技术生成

import urllib.request



#获取一个get请求
# response    = urllib.request.urlopen("http://www.baidu.com")
# print(response.read().decode('utf-8')) #对获取到的网页源码进行utf-8解码


#获取一个post请求
# import urllib.parse
# data = bytes(urllib.parse.urlencode({"hello":"world"}),encoding="utf-8")
# response = urllib.request.urlopen("http://httpbin.org/post",data= data)
# print(response.read().decode("utf-8"))



#超时处理
# try:
#     response = urllib.request.urlopen("http://httpbin.org/get", timeout=0.01)
#     print(response.read().decode("utf-8"))
# except  urllib.error.URLError as e:
#     print("time out!")


# response = urllib.request.urlopen("http://baidu.com")
# # print(response.status)
# print(response.getheaders())

# url = "https://www.douban.com"

# url = "http://httpbin.org/post"
# headers ={"User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.164 Safari/537.36"}
# data = bytes(urllib.parse.urlencode({'name':'eric'}),encoding="utf-8")
# req = urllib.request.Request(url= url,data=data,headers=headers,method="POST")
# response = urllib.request.urlopen(req)
# print(response.read().decode("utf-8"))


url = "https://www.douban.com"
headers ={"User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.164 Safari/537.36"}
req = urllib.request.Request(url= url,headers=headers)
response = urllib.request.urlopen(req)
print(response.read().decode("utf-8"))