爬取某地天气存入数据库

最新推荐文章于 2023-09-12 14:54:37 发布

反余弦函数

最新推荐文章于 2023-09-12 14:54:37 发布

阅读量307

点赞数

分类专栏： python爬虫文章标签：数据库 python

本文链接：https://blog.csdn.net/cp1024/article/details/128574826

版权

python爬虫专栏收录该内容

21 篇文章 2 订阅

订阅专栏

爬取某地天气存入数据库

import requests
import pymysql
import json
from lxml import etree

db = pymysql.connect(
    host='',
    user='root',
    password='',
    db='weather',
    port=,
    charset='utf8',
    # autocommit=1,
)

url = "http://weather.cma.cn/api/now/需要某个城市的id"
urls = "http://weather.cma.cn/web/weather/需要某个城市的id.html"
headers = {
    "User-Agent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0',
    "Referer": 'http://weather.cma.cn/web/weather/58321.html'
}
resp = requests.get(url=url, headers=headers)
resps = requests.get(url=urls, headers=headers)
resps.encoding = resps.apparent_encoding
tree = etree.HTML(resps.text)
weather = str(tree.xpath("//div[@id='dayList']/div/div[3]/text()")).replace('\\n', '').replace(' ', '').replace('\'','')[1]
datas = json.loads(resp.text)['data']['now']
# print(json.loads(resp.text))
# 降水量
precipitation = datas['precipitation']
# 温度
temperature = datas['temperature']
# 气压
pressure = datas['pressure']
# 湿度
humidity = datas['humidity']
# 风向
windDirection = datas['windDirection']
# 风向等级
windDirectionDegree = datas['windDirectionDegree']
# 风速
windSpeed = datas['windSpeed']
# 风速类型
windScale = datas['windScale']
lastUpdate = str(json.loads(resp.text)['data']['lastUpdate']).replace('/', '-')
sql = '''insert into yichang_weather(`precipitation`, `temperature`, `pressure`, `humidity`, `windDirection`, `windDirectionDegree`, `windSpeed`, `windScale`, `lastUpdate`,`weather`) values(\"{}\",\"{}\",\"{}\",\"{}\",\"{}\",\"{}\",\"{}\",\"{}\",\"{}\",\"{}\")'''.format(
    precipitation, temperature, pressure, humidity, windDirection, windDirectionDegree, windSpeed, windScale,
    lastUpdate, weather)
print(sql)
cursor = db.cursor()
cursor.execute(sql)
db.commit()
cursor.close()
db.close()