python保存文件到本地_Python爬虫爬取小说，并保存至本地文本文件中

运行环境：Python3.6、Pycharm2017.3、Windows10 RS3、requests2.18.4、BeautifulSoup4.6.0

简介

运行效果展示

① book136_singleprocess.py

单进程保存小说章节

#!/usr/bin/env python

# -*- coding: utf-8 -*-

# @Author : Woolei

# @File : book136_singleprocess.py

import requests

import time

import os

from bs4 import BeautifulSoup

headers = {

'User-Agent':

'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36'

}

# 获取小说章节内容，并写入文本

def getChapterContent(each_chapter_dict):

content_html = requests.get(each_chapter_dict['chapter_url'], headers=headers).text

soup = BeautifulSoup(content_html, 'lxml')

content_tag = soup.find('p', {'id': 'content'})

p_tag = content_tag.find_all('p')

print('正在保存的章节 -->