版权声明:未经允许,随意转载,请附上本文链接谢谢(づ ̄3 ̄)づ╭❤~
https://blog.csdn.net/xiaoduan_/article/details/80835248
豆瓣书评爬虫
一年前学的爬虫,忘得差不多了,现在复习一下
最近读了《白夜行》,就拿它下手喽
import requests
from lxml import etree
import pandas as pd
import time
data = []
def get_userdata(page):
for i in range(1, page + 1):
url = 'https://book.douban.com/subject/3259440/comments/hot?p={}'.format(i)
print("正在爬取第{}页\n".format(i))
print(time.strftime("%Y-%m-%d %H:%M:%S", time.localtime()))
time.sleep(1)
r = requests.get(url)
s = etree.HTML(r.text)
xiaodata = s.xpath('//div[@id="comments"]//div/p/text()')
data.extend(xiaodata)
if __name__ == '__main__':
get_userdata(20)
df = pd.DataFrame(data)
df.to_csv('白夜行.csv', encoding='utf-8-sig')
print("任务完成")
最后结果,爬20页意思意思就好啦