爬虫之盗亦有道简单实例

简单爬取京东商品信息

iimport requests
try:
    url="https://item.jd.com/100008348542.html"
    kv={'user-agent':'Mozilla/5.0'}
    r=requests.get(url,headers=kv)
    r.raise_for_status()
    r.enciding=r.apparent_encoding
    print(r.text[1000:2000])
    print(r.request.headers)
except:
    print( "产生异常")

爬取百度搜索信息

try:
    keyword="Python"
    kv={'wd':keyword}
    r=requests.get("http://www.baidu.com/s",params=kv)
    print(r.request.url)
    r.raise_for_status()
    print(len(r.text))
except:
    print("爬取失败")

爬取国家地图图片

import os 
url="https://www.nationalgeographic.com/animals/2020/07/lynx-long-journeys-mountains-alaska-canada/#/lynx-mather-03.jpg"
root="E//"
path=root+url.split("/")[-1]
try:
	if not os.path.exists(root):
		os.mkdir(root)
	if not os.path.exists(path):
		r=requests.get(url)
		with open(path,"wb") as f:
			f.write(r.content)
			f.close（）
			print("文件保存成功")
	else:
		print("文件已存在") 
except:
	print("爬取失败")

部分来自北京理工大学-嵩天的爬虫课程