Linux_Pip（Requests, Selenium, Scrapy等） 安装笔记

环境：Vmware Workstation 10，CentOS-7-x86_64-DVD-1511.iso，Xshell 4.0，ip：192.168.216.135.

• 注意：–是两个-， csdn显示问题。

[root@bogon ~]# wget https://bootstrap.pypa.io/get-pip.py –no-check-certificate

[root@bogon ~]# python get-pip.py

Collecting pip
Downloading pip-9.0.1-py2.py3-none-any.whl (1.3MB)
100% |████████████████████████████████| 1.3MB 51kB/s
Collecting setuptools
Downloading setuptools-36.6.0-py2.py3-none-any.whl (481kB)
100% |████████████████████████████████| 481kB 66kB/s
Collecting wheel
Downloading wheel-0.30.0-py2.py3-none-any.whl (49kB)
100% |████████████████████████████████| 51kB 55kB/s
Installing collected packages: pip, setuptools, wheel
Successfully installed pip-9.0.1 setuptools-36.6.0 wheel-0.30.0

[root@bogon ~]# pip -V

pip 9.0.1 from /usr/lib/python2.7/site-packages (python 2.7)

[root@bogon ~]# pip2 -V

pip 9.0.1 from /usr/lib/python2.7/site-packages (python 2.7)

[root@bogon ~]# pip3 -V

pip 9.0.1 from /usr/local/python3.6/site-packages (python 3.6)
至此pip安装成功

---

接下来使用pip安装requests，selenium和scrapy。

[root@bogon ~]# pip install requests

Collecting requests
Downloading requests-2.18.4-py2.py3-none-any.whl (88kB)
100% |████████████████████████████████| 92kB 110kB/s
Collecting chardet<3.1.0,>=3.0.2 (from requests)
Downloading chardet-3.0.4-py2.py3-none-any.whl (133kB)
100% |████████████████████████████████| 143kB 113kB/s
Collecting certifi>=2017.4.17 (from requests)
Downloading certifi-2017.7.27.1-py2.py3-none-any.whl (349kB)
100% |████████████████████████████████| 358kB 51kB/s
Collecting urllib3<1.23,>=1.21.1 (from requests)
Downloading urllib3-1.22-py2.py3-none-any.whl (132kB)
100% |████████████████████████████████| 133kB 41kB/s
Collecting idna<2.7,>=2.5 (from requests)
Downloading idna-2.6-py2.py3-none-any.whl (56kB)
100% |████████████████████████████████| 61kB 62kB/s
Installing collected packages: chardet, certifi, urllib3, idna, requests
Successfully installed certifi-2017.7.27.1 chardet-3.0.4 idna-2.6 requests-2.18.4 urllib3-1.22

---

pip 安装selenium

[root@bogon ~]# pip install selenium

Installing collected packages: selenium
Successfully installed selenium-3.6.0

[root@bogon ~]# wget http://npm.taobao.org/mirrors/chromedriver/2.33/chromedriver_linux64.zip

[root@bogon ~]# unzip chromedriver_linux64.zip -d /usr/bin/

Archive: chromedriver_linux64.zip
inflating: /usr/bin/chromedriver

[root@bogon ~]# cd /usr/bin/ && ls chromedriver

chromedriver

缺少chrome浏览器。

---

pip 安装scrapy
第一步下载和安装 pywin32，linux环境跳过此步骤。

[root@bogon ~]# yum install gcc python-devel

[root@bogon ~]# pip install scrapy

Building wheels for collected packages: Twisted
Running setup.py bdist_wheel for Twisted … done
Stored in directory: /root/.cache/pip/wheels/91/c7/95/0bb4d45bc4ed91375013e9b5f211ac3ebf4138d8858f84abbc
Successfully built Twisted
Installing collected packages: PyDispatcher, Twisted, scrapy
Successfully installed PyDispatcher-2.0.5 Twisted-17.9.0 scrapy-1.4.0

---

还可以安装aiohttp、beautifulsoup4、pyquery、pymysql、pymongo、redis、flask、django、jupyter、scrapy-redis等。

[root@bogon ~]# pip install aiohttp beautifulsoup4 pyquery pymysql pymongo redis flask django jupyter

[root@bogon ~]# pip –trusted-host pypi.python.org install scrapy-redis

安装splash

[root@bogon ~]# yum install docker
[root@bogon ~]# systemctl start docker
[root@bogon ~]# systemctl status docker
[root@bogon ~]# systemctl enable docker
[root@bogon ~]# docker pull scrapinghub/splash
[root@bogon ~]# docker run -p 8050:8050 -p 8051:8051 scrapinghub/splash
[root@bogon ~]# pip install scrapy-splash

Collecting scrapy-splash
Downloading scrapy_splash-0.7.2-py2.py3-none-any.whl
Installing collected packages: scrapy-splash
Successfully installed scrapy-splash-0.7.2