Linux下安装pyspider的详细过程和相关指令【无总结版】

最新推荐文章于 2022-01-08 16:35:02 发布
最新推荐文章于 2022-01-08 16:35:02 发布
阅读量5.9k 收藏
点赞数
分类专栏： pyspider 文章标签： pyspider linux 安装
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/myfancysky/article/details/48848331
版权
本文记录了在Linux系统中安装pyspider的详细过程，包括使用pip安装时遇到的问题及解决方法，如依赖库的安装，如python-devel, libffi-devel, openssl-devel等，并最终成功安装pyspider及其相关依赖。" 114823712,9482934,Python接口自动化测试实践：pytest+requests+pymysql+xlrd+allure,"['接口测试', '自动化测试', 'Python', 'pytest框架', '数据处理']
摘要由CSDN通过智能技术生成

下面是安装pyspider的详细过程及相关指令【纪实】
-----------------------------------------------------------------------------------------------------------------------------------------------------
*****************************************************************************************************************************************************
-----------------------------------------------------------------------------------------------------------------------------------------------------
[root@iZ28jyxu47dZ fancy]# pip install pyspider
Collecting pyspider
/usr/lib/python2.7/site-packages/pip-7.1.2-py2.7.egg/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  Downloading pyspider-0.3.5.tar.gz (94kB)
    100% |████████████████████████████████| 98kB 41kB/s
Collecting Flask>=0.10 (from pyspider)
  Downloading Flask-0.10.1.tar.gz (544kB)
    15% |████▉                           | 81kB 250bytes/s eta 0:30:44
  Hash of the package https://pypi.python.org/packages/source/F/Flask/Flask-0.10.1.tar.gz#md5=378670fe456957eb3c27ddaef60b2b24 (from https://pypi.python.org/simple/flask/) (e11c5569eb68d582ce1c85154b9b48c9) doesn't match the expected hash 378670fe456957eb3c27ddaef60b2b24!
Bad md5 hash for package https://pypi.python.org/packages/source/F/Flask/Flask-0.10.1.tar.gz#md5=378670fe456957eb3c27ddaef60b2b24 (from https://pypi.python.org/simple/flask/)

[root@iZ28jyxu47dZ fancy]# yum install python-devel libffi-devel openssl-devel
Loaded plugins: langpacks
base                                                     | 3.6 kB     00:00
epel                                                     | 4.3 kB     00:00
extras                                                   | 3.4 kB     00:00
updates                                                  | 3.4 kB     00:00
(1/7): epel/x86_64/group_gz                                | 169 kB   00:00
(2/7): base/7/x86_64/group_gz                              | 154 kB   00:00
(3/7): epel/x86_64/updateinfo                              | 361 kB   00:00
(4/7): extras/7/x86_64/primary_db                          |  87 kB   00:00
(5/7): epel/x86_64/primary_db                              | 3.5 MB   00:00
(6/7): base/7/x86_64/primary_db                            | 5.1 MB   00:01
(7/7): updates/7/x86_64/primary_db                         | 4.0 MB   00:00
Resolving Dependencies
--> Running transaction check
---> Package libffi-devel.x86_64 0:3.0.13-11.el7 will be installed
---> Package openssl-devel.x86_64 1:1.0.1e-42.el7.9 will be installed
--> Processing Dependency: openssl-libs(x86-64) = 1:1.0.1e-42.el7.9 for package: 1:openssl-devel-1.0.1e-42.el7.9.x86_64
--> Processing Dependency: zlib-devel(x86-64) for package: 1:openssl-devel-1.0.1e-42.el7.9.x86_64
--> Processing Dependency: krb5-devel(x86-64) for package: 1:openssl-devel-1.0.1e-42.el7.9.x86_64
---> Package python-devel.x86_64 0:2.7.5-18.el7_1.1 will be installed
--> Processing Dependency: python(x86-64) = 2.7.5-18.el7_1.1 for package: python-devel-2.7.5-18.el7_1.1.x86_64
--> Running transaction check
---> Package krb5-devel.x86_64 0:1.12.2-15.el7_1 will be installed
--> Processing Dependency: krb5-libs(x86-64) = 1.12.2-15.el7_1 for package: krb5-devel-1.12.2-15.el7_1.x86_64
--> Processing Dependency: libverto-devel for package: krb5-devel-1.12.2-15.el7_1.x86_64
--> Processing Dependency: libselinux-devel for package: krb5-devel-1.12.2-15.el7_1.x86_64
--> Processing Dependency: libcom_err-devel for package: krb5-devel-1.12.2-15.el7_1.x86_64
--> Processing Dependency: keyutils-libs-devel for package: krb5-devel-1.12.2-15.el7_1.x86_64
--> Processing Dependency: libkadm5srv_mit.so.9()(64bit) for package: krb5-devel-1.12.2-15.el7_1.x86_64
---> Package openssl-libs.x86_64 1:1.0.1e-34.el7_0.7 will be updated
--> Processing Dependency: openssl-libs(x86-64) = 1:1.0.1e-34.el7_0.7 for package: 1:openssl-1.0.1e-34.el7_0.7.x86_64
---> Package openssl-libs.x86_64 1:1.0.1e-42.el7.9 will be an update
---> Package python.x86_64 0:2.7.5-16.el7 will be updated
---> Package python.x86_64 0:2.7.5-18.el7_1.1 will be an update
--> Processing Dependency: python-libs(x86-64) = 2.7.5-18.el7_1.1 for package: python-2.7.5-18.el7_1.1.x86_64
---> Package zlib-devel.x86_64 0:1.2.7-13.el7 will be installed
--> Running transaction check
---> Package keyutils-libs-devel.x86_64 0:1.5.8-3.el7 will be installed
---> Package krb5-libs.x86_64 0:1.11.3-49.el7 will be updated
---> Package krb5-libs.x86_64 0:1.12.2-15.el7_1 will be an update
---> Package libcom_err-devel.x86_64 0:1.42.9-7.el7 will be installed
--> Processing Dependency: libcom_err(x86-64) = 1.42.9-7.el7 for package: libcom_err-devel-1.42.9-7.el7.x86_64
---> Package libselinux-devel.x86_64 0:2.2.2-6.el7 will be installed
--> Processing Dependency: libsepol-devel >= 2.1.9-1 for package: libselinux-devel-2.2.2-6.el7.x86_64
--> Processing Dependency: pkgconfig(libsepol) for package: libselinux-devel-2.2.2-6.el7.x86_64
--> Processing Dependency: pkgconfig(libpcre) for package: libselinux-devel-2.2.2-6.el7.x86_64
---> Package libverto-devel.x86_64 0:0.2.5-4.el7 will be installed
---> Package openssl.x86_64 1:1.0.1e-34.el7_0.7 will be updated
---> Package openssl.x86_64 1:1.0.1e-42.el7.9 will be an update
---> Package python-libs.x86_64 0:2.7.5-16.el7 will be updated
---> Package python-libs.x86_64 0:2.7.5-18.el7_1.1 will be an update
--> Running transaction check
---> Package libcom_err.x86_64 0:1.42.9-4.el7 will be updated
--> Processing Dependency: libcom_err(x86-64) = 1.42.9-4.el7 for package: e2fsprogs-1.42.9-4.el7.x86_64
--> Processing Dependency: libcom_err(x86-64) = 1.42.9-4.el7 for package: e2fsprogs-libs-1.42.9-4.el7.x86_64
--> Processing Dependency: libcom_err(x86-64) = 1.42.9-4.el7 for package: libss-1.42.9-4.el7.x86_64
---> Package libcom_err.x86_64 0:1.42.9-7.el7 will be an update
---> Package libsepol-devel.x86_64 0:2.1.9-3.el7 will be installed
---> Package pcre-devel.x86_64 0:8.32-14.el7 will be installed
--> Processing Dependency: pcre(x86-64) = 8.32-14.el7 for package: pcre-devel-8.32-14.el7.x86_64
--> Running transaction check
---> Package e2fsprogs.x86_64 0:1.42.9-4.el7 will be updated
---> Package e2fsprogs.x86_64 0:1.42.9-7.el7 will be an update
---> Package e2fsprogs-libs.x86_64 0:1.42.9-4.el7 will be updated
---> Package e2fsprogs-libs.x86_64 0:1.42.9-7.el7 will be an update
---> Package libss.x86_64 0:1.42.9-4.el7 will be updated
---> Package libss.x86_64 0:1.42.9-7.el7 will be an update
---> Package pcre.x86_64 0:8.32-12.el7 will be updated
---> Package pcre.x86_64 0:8.32-14.el7 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package                  Arch        Version                Repository    Size
================================================================================
Installing:
 libffi-devel             x86_64      3.0.13-11.el7          base          22 k
 openssl-devel            x86_64      1:1.0.1e-42.el7.9      updates      1.2 M
 python-devel             x86_64      2.7.5-18.el7_1.1       updates      389 k
Installing for dependencies:
 keyutils-libs-devel      x86_64      1.5.8-3.el7            base          37 k
 krb5-devel               x86_64      1.12.2-15.el7_1        updates      641 k
 libcom_err-devel         x86_64      1.42.9-7.el7           base          30 k
 libselinux-devel         x86_64      2.2.2-6.el7            base         174 k
 libsepol-devel           x86_64      2.1.9-3.el7            base          71 k
 libverto-devel           x86_64      0.2.5-4.el7            base          12 k
 pcre-devel               x86_64      8.32-14.el7            base         477 k
 zlib-devel               x86_64      1.2.7-13.el7           base          49 k
Updating for dependencies:
 e2fsprogs                x86_64      1.42.9-7.el7           base         697 k
 e2fsprogs-libs           x86_64      1.42.9-7.el7           base         165 k
 krb5-libs                x86_64      1.12.2-15.el7_1        updates      826 k
 libcom_err               x86_64      1.42.9-7.el7           base          40 k
 libss                    x86_64      1.42.9-7.el7           base          45 k
 openssl                  x86_64      1:1.0.1e-42.el7.9      updates      711 k
 openssl-libs             x86_64      1:1.0.1e-42.el7.9      updates      949 k
 pcre                     x86_64      8.32-14.el7            base         418 k
 python                   x86_64      2.7.5-18.el7_1.1       updates       86 k
 python-libs              x86_64      2.7.5-18.el7_1.1       updates      5.6 M

Transaction Summary
================================================================================
Install  3 Packages (+ 8 Dependent packages)
Upgrade             ( 10 Dependent packages)

Total download size: 12 M
Is this ok [y/d/N]: y
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
(1/21): e2fsprogs-libs-1.42.9-7.el7.x86_64.rpm             | 165 kB   00:00
(2/21): keyutils-libs-devel-1.5.8-3.el7.x86_64.rpm         |  37 kB   00:00
(3/21): libcom_err-1.42.9-7.el7.x86_64.rpm                 |  40 kB   00:00
(4/21): libcom_err-devel-1.42.9-7.el7.x86_64.rpm           |  30 kB   00:00
(5/21): libffi-devel-3.0.13-11.el7.x86_64.rpm              |  22 kB   00:00
(6/21): libselinux-devel-2.2.2-6.el7.x86_64.rpm            | 174 kB   00:00
(7/21): libsepol-devel-2.1.9-3.el7.x86_64.rpm              |  71 kB   00:00
(8/21): e2fsprogs-1.42.9-7.el7.x86_64.rpm                  | 697 kB   00:00
(9/21): libss-1.42.9-7.el7.x86_64.rpm                      |  45 kB   00:00
(10/21): libverto-devel-0.2.5-4.el7.x86_64.rpm             |  12 kB   00:00
(11/21): krb5-devel-1.12.2-15.el7_1.x86_64.rpm             | 641 kB   00:00
(12/21): openssl-1.0.1e-42.el7.9.x86_64.rpm                | 711 kB   00:00
(13/21): krb5-libs-1.12.2-15.el7_1.x86_64.rpm              | 826 kB   00:00
(14/21): openssl-libs-1.0.1e-42.el7.9.x86_64.rpm           | 949 kB   00:00
(15/21): openssl-devel-1.0.1e-42.el7.9.x86_64.rpm          | 1.2 MB   00:00
(16/21): python-2.7.5-18.el7_1.1.x86_64.rpm                |  86 kB   00:00
(17/21): python-devel-2.7.5-18.el7_1.1.x86_64.rpm          | 389 kB   00:00
(18/21): pcre-devel-8.32-14.el7.x86_64.rpm                 | 477 kB   00:00
(19/21): zlib-devel-1.2.7-13.el7.x86_64.rpm                |  49 kB   00:00
(20/21): pcre-8.32-14.el7.x86_64.rpm                       | 418 kB   00:00
(21/21): python-libs-2.7.5-18.el7_1.1.x86_64.rpm           | 5.6 MB   00:01
--------------------------------------------------------------------------------
Total                                              5.1 MB/s |  12 MB  00:02
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : libcom_err-1.42.9-7.el7.x86_64                              1/31
  Updating   : 1:openssl-libs-1.0.1e-42.el7.9.x86_64                       2/31
  Updating   : krb5-libs-1.12.2-15.el7_1.x86_64                            3/31
  Updating   : python-libs-2.7.5-18.el7_1.1.x86_64                         4/31
  Updating   : python-2.7.5-18.el7_1.1.x86_64                              5/31
  Updating   : e2fsprogs-libs-1.42.9-7.el7.x86_64                          6/31
  Updating   : libss-1.42.9-7.el7.x86_64                                   7/31
  Installing : libcom_err-devel-1.42.9-7.el7.x86_64                        8/31
  Installing : zlib-devel-1.2.7-13.el7.x86_64                              9/31
  Updating   : pcre-8.32-14.el7.x86_64                                    10/31
  Installing : pcre-devel-8.32-14.el7.x86_64                              11/31
  Installing : libverto-devel-0.2.5-4.el7.x86_64                          12/31
  Installing : libsepol-devel-2.1.9-3.el7.x86_64                          13/31
  Installing : libselinux-devel-2.2.2-6.el7.x86_64                        14/31
  Installing : keyutils-libs-devel-1.5.8-3.el7.x86_64                     15/31
  Installing : krb5-devel-1.12.2-15.el7_1.x86_64                          16/31
  Installing : 1:openssl-devel-1.0.1e-42.el7.9.x86_64                     17/31
  Updating   : e2fsprogs-1.42.9-7.el7.x86_64                              18/31
  Installing : python-devel-2.7.5-18.el7_1.1.x86_64                       19/31
  Updating   : 1:openssl-1.0.1e-42.el7.9.x86_64                           20/31
  Installing : libffi-devel-3.0.13-11.el7.x86_64                          21/31
  Cleanup    : 1:openssl-1.0.1e-34.el7_0.7.x86_64                         22/31
  Cleanup    : e2fsprogs-1.42.9-4.el7.x86_64                              23/31
  Cleanup

最低0.47元/天 解锁文章
fancyChuan2
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 2
    评论
专栏目录
Linux/ubuntu 下安装pyspider的过程
09-15
pyspider 是 python 语言的一个爬虫工具，提供有GUI界面，可以在web界面中制定任务、爬取数据和导出结果。这篇文章主要介绍了Linux/ubuntu 下安装pyspider的过程,需要的朋友可以参考下
windows安装pyspider环境
09-02
包含如下依赖环境： python-2.7.12.amd64 lxml-2.2.8.win-amd64-py2.7 ez_setup-0.9.tar pip-8.1.2.tar 结合我的blog文章使用： http://blog.csdn.net/zengraoli/article/details/52415139
LINUX搭建PySpider爬虫服务
weixin_30394633的博客
09-26 182
1.环境搭建 yum update -y yum install gcc gcc-c++ -y yum install python-pip python-devel python-distribute libxml2 libxml2-devel python-lxml libxslt libxslt-devel openssl openssl-devel -y pip install...
Python学习----pyspider（linux）（一）
weixin_40132006的博客
09-18 329
官方文档：http://docs.pyspider.org/en/latest/ Anaconda使用请参考：https://blog.csdn.net/weixin_40132006/article/details/108629324 pyspider安装 pip install pyspider 安装经历（Anaconda下） 在python3.8版本下 pip install pyspider //出现错误 (base) bobo@bobo-PC:~$ pyspider Traceb
pyspider在Linux的安装和基本使用
weixin_43326122的博客
05-15 842
安装pyspider python==2.7 升级pip pip install --upgrade pip 安装pyspider pip install pyspider 安装依赖 yum install bzip2 yum install fontconfig yum install curl pip install mysql-connector pip install redis 开放端口 phantomjs 25555 pyspider 5000 安装phantomjs https://ph
PySpider
代码改变世界
05-31 352
   
Python3爬虫中pyspider的安装步骤
01-21
pyspider是国人binux编写的强大的网络爬虫框架，它带有强大的WebUI、脚本编辑器、任务监控器、项目管理器以及结果处理器，同时支持多种数据库后端、多种消息队列，另外还支持JavaScript渲染页面的爬取，使用起来非常...
python3.7+安装pyspider报错解决方案.docx
10-26
### Python 3.7+ 安装 PySpider 报错解决方案 #### 环境配置 在开始之前，我们先了解下本次问题解决所处的环境背景，这有助于更好地定位和解决问题。 - **操作系统**: macOS - **Python 版本**: 3.7 #### 遇到的...
CentOS 7最小化安装之后的设置
xiayige_19831001的博客
02-05 1436
Last login: Tue Feb  5 13:25:05 2019 [tedu@localhost ~]$ su 密码： [root@localhost tedu]# service network restart Restarting network (via systemctl):                        [  确定  ] [root@localhost tedu]...
Requirement already satisfied
热门推荐
qq_43767511的博客
01-08 2万+
Requirement already satisfied: 今天下载python包时候碰到下面问题： Requirement already satisfied: numpy in c:\users\lenovo\appdata\roaming\python\python36\site-packages (1.19.4) 无论下载什么包都报相同错误，翻译过来 “要求已经满足”，看了其他人写的文章，归结几点原因 一. 网络原因，查看一下镜像源，可能没有添加相应的镜像源，在不同的软件下可以查寻具体添加，Py
Python pyspider的安装
Donnie_Song的博客
05-13 3437
Python pyspider的安装前言一、平台二、Python版本三、安装pyspider四、执行 前言 ： 例如：随着人工智能的不断发展，机器学习这门技术也越来越重要，很多人都开启了学习机器学习，本文就介绍了机器学习的基础内容。 提示：以下是本篇文章正文内容，下面案例可供参考 一、平台 本机使用WIN 10平台进行配置 二、Python版本 本机使用Python版本为3.6.2（不建议使用高版本，有些库会不兼容），下载链接，使用如下截图中的安装包安装即可。 三、安装pyspider 打开cmd，直
pyspider常见问题与解决方案
黄油猫
02-26 2302
1.HTTPError: HTTP 599: SSL certificate problem: self signed certificate in certi…这个问题， 我们采用的是忽略证书， 为crawl 方法添加参数 validate_cert = False, 即可2.css 选择器pyspider 可以匹配相应的字符串， 支持css 选择器功能， 但是需要注意的是 在chrome 中 ...
记一次Linux下安装pyspider的过程
FK103的专栏
04-10 1597
首先执行pip install pyspider此时系统提示Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-Lau0Qp/pycurl/ You are using pip version 9.0.1, however version 9.0.3 is available. You shou...
docker安装gitlab只需要3分钟
weixin_33834075的博客
07-25 198
docker安装gitlab sudo docker run --detach --hostname 127.0.0.1 --publish 1443:443 --publish 180:80 --publish 122:22 --name gitlab --volume /data/gi...
redis集群 部署操作流程
u011411602的博客
04-20 4263
redis集群部署搭建方法
Found existing installation:xxxx
qq_43139020的博客
06-29 2792
Found existing installation:xxxx 最近在跑模型的时候老遇到这种问题： Found existing installation: PyYAML 5.3.1 ERROR: Cannot uninstall ‘PyYAML’. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a p
pyspider的使用
小岁月太着急
08-01 2万+
原文地址 pyspider 是一个用python实现的功能强大的网络爬虫系统，能在浏览器界面上进行脚本的编写，功能的调度和爬取结果的实时查看，后端使用常用的数据库进行爬取结果的存储，还能定时设置任务与任务优先级等。 本篇文章只是对这个框架使用的大体介绍，更多详细信息可见官方文档。 安装 首先是环境的搭建，网上推荐的各种安装命令，如： pip install pyspi
linux下的pyspider怎么配置连接自己的mysql数据库
最新发布
04-26
要在Linux下配置pyspider连接到自己的MySQL数据库，需要进行以下几个步骤： 1. 安装MySQL数据库，并创建一个新的数据库和用户，并为该用户分配合适的权限。例如： ``` CREATE DATABASE pyspider_db; CREATE USER '...
评论 2
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值