今天把本地运行OK的scrapy爬虫程序捣鼓到服务器上运行,结果报了以下错误
2018-10-11 19:00:19 [twisted] CRITICAL: Unhandled error in Deferred:
2018-10-11 19:00:19 [twisted] CRITICAL:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
result = g.send(result)
File "/usr/local/lib/python3.7/site-packages/scrapy/crawler.py", line 82, in crawl
yield self.engine.open_spider(self.spider, start_requests)
ModuleNotFoundError: No module named '_sqlite3'
网上搜说sqlite3模块没有,需要安装并重新编译,然而网上各种方式都无效,好坑,最后在stackoverflow上找到个有效的方式
第一步安装 sqlite-devel
第二步 cd到python3源码目录 运行./configure --enable-loadable-sqlite-extensions && make && sudo make install