豆瓣爬虫+保存到csv+生成词云+数据可视化+封面下载

最新推荐文章于 2024-01-17 17:06:16 发布

只想可爱下去

最新推荐文章于 2024-01-17 17:06:16 发布

阅读量368

点赞数

分类专栏：爬虫文章标签：爬虫 python 数据可视化 csv

本文链接：https://blog.csdn.net/qq_45151080/article/details/113375889

版权

该博客介绍了如何使用Python进行豆瓣数据爬取，将获取的数据保存到CSV文件，并进一步进行了词云生成及数据可视化，同时展示了如何下载封面图片。通过实例展示了爬虫、数据处理和可视化的基本流程。

摘要由CSDN通过智能技术生成

代码运行结果图

代码区

import csv

from urllib import request
import requests
from wordcloud import WordCloud
import matplotlib.pyplot as plt



def Spider():
    headers = {
   
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
    }
    mfile = open("db.csv", "w", encoding="utf-8", newline="", errors="ignore")
    writer = csv.writer(mfile)
    writer.writerow(["编号", "电视剧名字", "评分", "是否新上映", "封面", "播放源", ])

    mcount = 1

    for i in range(1, 21):
        url = "https://movie.douban.com/j/search_subjects?type=tv&tag=%E7%83%AD%E9%97%A8&sort=recommend&page_limit=20&page_start=" + str(i * 20)
        mjson = requests.get(url, headers=headers).json()
        for j in mjson['subjects']:
            name = j['title']
            rate = j['rate']
            is_new = j['is_new']
            cover = j['cover']
            player = j['url']
            writer.writerow([str(mcount), name, rate, is_new, cover, player])