python scrapy框架例子

最新推荐文章于 2024-04-14 14:41:59 发布

_fox

最新推荐文章于 2024-04-14 14:41:59 发布

阅读量63

点赞数

文章标签： python scrapy windows

本文链接：https://blog.csdn.net/qq_19599237/article/details/134129649

版权

1.数据解析

import scrapy
class QiubaiSpider(scrapy.Spider):
    name = "qiubai"
    #一般注释掉这个
    #allowed_domains = ["www.xxx.com"]
    start_urls = ["https://movie.douban.com/review/best/"]

    def parse(self, response):
        div_list = response.xpath('//div[@class="review-list chart "]/div')
        for div in div_list:
            # extract将selector对象中data参数存储的字符串提取出来
            comment = div.xpath('//div[@class="short-content"]').extract()
            #列表转为字符串
            comment = ''.join(comment)
            print(comment)