Java 爬虫 x Spring Boot

Porn Bot：Pornhub-downloader Link

介紹

• 安装简单，所有相依的套件已包含，只需有Java环境。
• 爬虫支援多Thread, 可24小时持续爬取
• Spring boot x H2 Db 纪录已爬过的网站
• 解决了该网站429防爬虫status

github連結 喜欢希望您能不吝给予星星, 笔者将会持续撰写更多优质的文章

Environment, Architecture

• Java1.8

• Crawler4j

• Spring Boot x H2 Db

Run

java -jar PornBot.jar

Database Description

http://localhost:8000/h2-console/

JDBC URL: jdbc:h2:~/porn/porn-db

User Name: sa

Password: empty

Record Table：

 Table_Name               ：PORN_RECORD
 viewKey                  ：The website's video unique key.
 imageUrl                 ：Image url of video.
 linkUrl                  ：Video jump to Website`s link
 videoUrl                 ：Video adrress.
 videoTitle               ：Title of video.
 videoDuration            ：Video click count.
 videoQuality             ：Defualt quality - 240, 480, 960, 1280p.
 download                 ：Has been downloaded. True or false.
 createdTime              ：The record created time.
 filePath                 ：The video downloaded path.
复制代码

更多設定請見 See Default Configuration