python爬虫小案例_从豆瓣爬取《哪吒之魔童降世》的短评-CSDN博客

本文链接：https://blog.csdn.net/Dina_p/article/details/100747805

一、在豆瓣电影网爬取影片的短评以及相关的信息：

二、参考代码

import requests
from bs4 import BeautifulSoup
import pandas as pd

url_list = []
for i in range(5):
    url_list.append('https://movie.douban.com/subject/26794435/comments?start=%s&limit=20&sort=new_score&status=P'%(i*20))

n = 0
commentInfo_list =[]
for url in url_list:
    txt = requests.get(url).text # 网页的获取
    soup = BeautifulSoup(txt,'lxml')  # 网页的解析

    divs = soup.find('div',id='comments').find_all('div',class_='comment')        
    for div in divs:
        comments_dict ={}        
        comments_dict['用户名'] = div.find('span',class_ = 'comment-info').find('a').text
        comments_dict['评论时间'] = div.find('span',class_= 'comment-time').text.replace(' ','').replace('\n','')
        comments_dict['有用数量'] = div.find('span',class_= 'votes').text
        comments_dict['评论内容'] = div.find('span',class_='short').text
        
        commentInfo_list.append(comments_dict)            
        n += 1        
print('已成功采集%s条评论'%n)    

df = pd.DataFrame(commentInfo_list)

三、查看爬取下来的短评以及相关的信息：