简单爬取豆瓣排行榜,并保存为json格式

import requests
import json
headers={
        "User-Agent": "Mozilla / 5.0(Windows NT 6.1;WOW64) AppleWebKit / 537.36(KHTML, likeGecko) Chrome / 85.0.4183.83Safari / 537.36"
    }
url = 'https://movie.douban.com/j/chart/top_list'
param = {
        "type": "24",
        "interval_id": "100:90",
        "action": "",
        "start": "20",
        "limit": "20",
}
response = requests.get(url=url,params=param,headers=headers)
list_data = response.content.decode()
filename = 'doubin.json'
with open(filename,'w',encoding='utf-8') as fp:
    fp.write(filename)

print(filename,'保存成功')

  • 1
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
以下是一个Python程序,用于爬取豆瓣图书Top250并将结果保存JSON文件: ```python import requests import json from bs4 import BeautifulSoup def get_book_info(url): headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'} response = requests.get(url, headers=headers) soup = BeautifulSoup(response.text, 'html.parser') book_list = soup.find('div', {'class': 'article'}).find_all('li') books = [] for book in book_list: title = book.find('div', {'class': 'pl2'}).find('a').text.strip() rating = book.find('span', {'class': 'rating_nums'}).text.strip() author_info = book.find('p', {'class': 'pl'}).text.strip().split('/') author = author_info[0].strip() publisher = author_info[-3].strip() pub_date = author_info[-2].strip() price = author_info[-1].strip() books.append({'title': title, 'rating': rating, 'author': author, 'publisher': publisher, 'pub_date': pub_date, 'price': price}) return books all_books = [] for i in range(0, 250, 25): url = 'https://book.douban.com/top250?start=' + str(i) books = get_book_info(url) all_books.extend(books) with open('douban_top250.json', 'w', encoding='utf-8') as f: json.dump(all_books, f, ensure_ascii=False, indent=4) ``` 该程序使用requests库和BeautifulSoup库来解析豆瓣图书Top250页面,并将结果保存JSON文件。首先定义了一个函数get_book_info来获取每一页的书籍信息,然后使用一个循环来遍历所有页面并将结果保存为一个列表。最后,使用json.dump函数将结果保存JSON文件。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值