本机环境:mac air、macOS12.2、python3.9.6
问题描述
使用scrapy crawl xxx
(xxx是我的项目名) 命令爬虫时出现错误:MemoryError: Cannot allocate write+execute memory for ffi.callback(). You might be running on a system that prevents this. For more information, see https://cffi.readthedocs.io/en/latest/using.html#callbacks
最终解决方案
只需将python版本升级为3.10.2
,然后重新安装scrapy,问题即可解决!!
参考自github的issue:
Figured it out following @anribras 's advice. M1 mac, had the same issue here. As @anribras says, I upgraded Python in my conda env to 3.10.2 and now this issue is resolved.
链接: github.
其他解决方案
1、更新requests库,我尝试后不行。
pip3 install --upgrade requests
参考链接:stackflow
2、删除pyopenssl库,由于我是因为使用scrapy库出现的这个错误,而scrapy库又依赖于pyopenssl库,所以我删除之后依然不行。
pip3 uninstall pyopenssl
参考链接:CSDN
参考链接:
1.https://github.com/pyca/pyopenssl/issues/873#issuecomment-1072969241
2.https://stackoverflow.com/questions/67556847/scrapy-on-m1-mac-memoryerror-cannot-allocate-writeexecute-memory-for-ffi-call
3.https://blog.csdn.net/hrbust_cxl/article/details/115283524