爬取笔趣阁部分小说及对应章节

最新推荐文章于 2021-06-28 20:21:13 发布

Z0o1010

最新推荐文章于 2021-06-28 20:21:13 发布

阅读量597

点赞数

分类专栏： Spider_Flask_HTML

本文链接：https://blog.csdn.net/weixin_48732879/article/details/109619339

版权

本文介绍如何使用Python爬虫技术从笔趣阁网站抓取部分小说的章节信息，包括GetSave.py用于爬取数据，Show.py用于展示结果，以及生成的HTML文件Show.html和Chapter.html展示爬取内容。

摘要由CSDN通过智能技术生成

GetSave.py

import requests
import time
from lxml import etree
from flask import Flask
from flask_sqlalchemy import SQLAlchemy

app = Flask(__name__)
app.config["SQLALCHEMY_DATABASE_URI"] = "mysql+pymysql://root:681053@localhost:3305/test?charset=utf8"
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = True
db = SQLAlchemy(app=app)


class Bookss(db.Model):
    __table_name__ = "bookss"
    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String(64))
    url = db.Column(db.String(256))
    b_c = db.relationship("Chapter", backref="bookss")


class Chapter(db.Model):
    __table_name__ = "chapter"
    id = db.Column(db.Integer, primary_key=True)
    chapter = db.Column(db.String(64))
    url = db.Column(db.String(256))
    b_id = db.Column(db.Integer, db.ForeignKey(Bookss.id))


def get(url):
    time.sleep(0.5)
    headers = {
   
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/"
                      "86.0.4240.111 Safari/537.36"
    }
    reponse = requests.get(url=url, headers=headers)
    if reponse