豆瓣读书top250数据爬取与可视化

爬虫–scrapy

题目：根据豆瓣读书top250,根据出版社对书籍数量分类，绘制饼图

搭建环境

import scrapy
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

加载scrapy框架

#terminal 终端实现
cd .. # 跳转到上一层目录
scrapy startproject booktop # 和项目同名的scrapy框架项目

setting配置

ROBOTSTXT_OBEY = False # 君子协议 false 不遵守
USER_AGENT = 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36'
DOWNLOAD_DELAY = 0.5 # 下载延迟## 如何改变文本的样式

spider编写

#spiders文件夹下创建python文件 bookspider.py
import scrapy
from booktop.items import BookItem
class BookSpider(scrapy.Spider):
		name="bookspider"
		al