【解决记录】Scrapy start_urls 出现NameError

 
scrap有 init 文件prod_list变量已经define了。但是怎么也不能正常运行


Error 信息:

 File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "D:\E-com\IC\jdcomment_total\jdcomment_total\spiders\__init__.py", line 16, in <module>
    class jdcommenttotal(scrapy.Spider):
  File "D:\E-com\IC\jdcomment_total\jdcomment_total\spiders\__init__.py", line 21, in jdcommenttotal
    range(0, len(prod_list), 10)]
  File "D:\E-com\IC\jdcomment_total\jdcomment_total\spiders\__init__.py", line 20, in <listcomp>
    start_urls = ['https://club.jd.com/clubservice/summary-m-' + (",").join(prod_list[i:i + 10]) + '.html' for i in
NameError: name 'prod_list' is not defined

prod_list = ['5075562', '1431097', '4189077', '4861396', '5247800', '10484104852', '18664498859', '10483921156',
                     '1020020079', '28679671154', '28471641171', '1654397634', '10348802024', '10348352167',
                     '28505858419', '28505712997', '28965267648', '12948467735', '28972238665', '28972035230',
                     '28972426881', '28972379220', '28972339165', '28972417946', '28972433168', '28972389531',
                     '28972395689', '28972393139', '1340987552']
start_urls = ['https://club.jd.com/clubservice/summary-m-' + (",").join(prod_list[i:i + 10]) + '.html' for i in range(0, len(prod_list), 10)]

找到了偏方可以正常解决。使用lambda 正常运行了。

 start_urls = list((map(lambda url: "https://club.jd.com/clubservice/summary-m-" + url + ".html", prod_list)))

但是我要保持每隔10个生成一组的list。单独define start_urls就可以用了

    def __init__(self):
        prod_list = ['5075562', '1431097', '4189077', '4861396', '5247800', '10484104852', '18664498859', '10483921156',
                     '1020020079', '28679671154', '28471641171', '1654397634', '10348802024', '10348352167',
                     '28505858419', '28505712997', '28965267648', '12948467735', '28972238665', '28972035230',
                     '28972426881', '28972379220', '28972339165', '28972417946', '28972433168', '28972389531',
                     '28972395689', '28972393139', '1340987552']
        self.start_urls = ['https://club.jd.com/clubservice/summary-m-' + (",").join(prod_list[i:i + 10]) + '.html' for i in
                  range(0, len(prod_list), 10)]


  • 1
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值