安装:
Linux/mac
- pip3 install scrapy
Windows:
- 安装twsited
a. pip3 install wheel
b. 下载twisted http://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted
c. 进入下载目录,执行 pip3 install Twisted-xxxxx.whl
- 安装scrapy
d. pip3 install scrapy -i http://pypi.douban.com/simple --trusted-host pypi.douban.com
- 安装pywin32
e. pip3 install pywin32 -i http://pypi.douban.com/simple --trusted-host pypi.douban.com
快速使用:
Django:
django-admin startproject mysite
cd mysite
python manage.py startapp app01
# 写代码
python manage.py runserver
Scrapy:
创建project:
scrapy startproject xianglong
cd xianglong
scrapy genspider chouti chouti.com
# 写代码
scrapy crawl chouti --nolog
遇到的坑
from cryptography.hazmat.bindings._constant_time import lib
ModuleNotFoundError: No module named ‘cryptography.hazmat.bindings._constant_time’
先卸载 pip uninstall cryptography
在安装 pip install paramiko
就可以了