爬虫之爬取小说

爬虫之爬取小说

展示

源码地址

https://github.com/Liar0320/spider

依赖

• cherrio Fast, flexible & lean implementation of core jQuery designed specifically for the server.
• inquirer A collection of common interactive command line user interfaces.

安装

Clone the repo using Git:

git clone https://github.com/Liar0320/spider.git

Alternatively you can download this repository and then:

npm i 
npm run start 

配置参数

请求的超时时间

Timeout for each request.

default value: 15000

最大同时下载多少章节

The maximum number of threads that exist at one time, that is, the number of asynchronous downloads that are started at the same time.

default value: 20

请求失败时，重新发起请求的次数

The number of times the request was re initiated if there was a request failure.

default value: 4

下载的目录

Root of download directory

default value: join(__dirname, ‘./download’)