一、介绍
本文介绍爬虫的一个小用法,爬取天气预报
写爬虫时最好更改一下User-Agent,不然会被阻止访问。
本文用正则匹配获取天气预报的数据,并使用matplotlib进行绘图,然后进行分析天气预报的数据
二、具体代码
import requests
from bs4 import BeautifulSoup
import re
import pandas as pd
import matplotlib.pyplot as plt
url = 'https://www.tianqi.com/guangdong-guilin/28722/30/'
headers = {
"User-Agent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/114.0'
}
response = requests.get(url, headers=headers)
response.encoding = 'utf-8'
html = response.text
soup = BeautifulSoup(html, 'html.parser') #解析文档,html.parser为HTML解析器
weather = []
tem = []
for tag in soup.find_all('li',attrs={"class":"temp"}):
if not bool(re.search(r