scrapy 安装数据库mysql错误汇总和操作步骤

最新推荐文章于 2021-01-21 11:28:57 发布

Java不是C

最新推荐文章于 2021-01-21 11:28:57 发布

阅读量1.9k

点赞数

分类专栏：网络爬虫

本文链接：https://blog.csdn.net/kangqianglong/article/details/53196926

版权

网络爬虫专栏收录该内容

14 篇文章 0 订阅

订阅专栏

　操作步骤：

　　1、检查python是否支持mysql

[root@bogon ~]# python
Python 2.7.10 (default, Jun  5 2015, 17:56:24) 
[GCC 4.4.4 20100726 (Red Hat 4.4.4-13)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import MySQLdb
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named MySQLdb

[root@bogon MySQL-python-1.2.5]# python
Python 2.7.12 (default, Oct 18 2016, 02:30:08)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import MySQLdb
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "MySQLdb/__init__.py", line 19, in <module>
import _mysql
ImportError: No module named _mysql
>>>

　　如果出现：ImportError: No module named MySQLdb则说明python尚未支持mysql，需要手工安装，请参考步骤2；如果没有报错，请调到步骤3

2、python安装mysql支持

[root@bogon ~]# pip install mysql-python
Collecting mysql-python
  Downloading MySQL-python-1.2.5.zip (108kB)
    100% |████████████████████████████████| 110kB 115kB/s 
Building wheels for collected packages: mysql-python
  Running setup.py bdist_wheel for mysql-python
  Stored in directory: /root/.cache/pip/wheels/8c/0d/11/d654cad764b92636ce047897dd2b9e1b0cd76c22f813c5851a
Successfully built mysql-python
Installing collected packages: mysql-python
Successfully installed mysql-python-1.2.5

安装的时候报错：

[root@bogon MySQL-python-1.2.5]# pip install mysql-python
Collecting mysql-python
Using cached MySQL-python-1.2.5.zip
Complete output from command python setup.py egg_info:
sh: mysql_config: command not found
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-hOhW5F/mysql-python/setup.py", line 17, in <module>
metadata, options = get_config()
File "setup_posix.py", line 43, in get_config
libs = mysql_config("libs_r")
File "setup_posix.py", line 25, in mysql_config
raise EnvironmentError("%s not found" % (mysql_config.path,))
EnvironmentError: mysql_config not found

----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-hOhW5F/mysql-python/
[root@bogon MySQL-python-1.2.5]#

　　安装完以后再次运行步骤1，检查python是否已经支持mysql

　　如果还有问题您可以尝试：LC_ALL=C pip install mysql-python
　　如果依然报错：error: Python.h: No such file or directory
　　您可以尝试先安装python-devel：

yum install python-devel

安装Python-devel 报错：

[root@bogon MySQL-python-1.2.5]# yum install mysql-devel 报错

mysql-5.1.71-1.el6.x86_64: failure: Packages/mysql-5.1.71-1.el6.x86_64.rpm from rhel-iso: [Errno 256] No more mirrors to try.
mysql-devel-5.1.71-1.el6.x86_64: failure: Packages/mysql-devel-5.1.71-1.el6.x86_64.rpm from rhel-iso: [Errno 256] No more mirrors to try.

[root@bogon MySQL-python-1.2.5]# rpm -qa | grep -i mysql
mysql-libs-5.1.71-1.el6.x86_64

下载安装包，手动安装：

[root@bogon MySQL-python-1.2.5]# ll
total 192
drwxr-xr-x. 3 root root 4096 Nov 16 01:41 doc
-rw-rw-rw-. 1 root root 18092 Oct 8 2012 GPL-2.0
-rw-rw-rw-. 1 root root 5687 Jan 1 2013 HISTORY
-rw-rw-rw-. 1 root root 6636 Oct 8 2012 INSTALL
-rw-rw-rw-. 1 root root 309 Jan 2 2014 MANIFEST.in
-rw-rw-rw-. 1 root root 2039 Jan 2 2014 metadata.cfg
-rw-rw-rw-. 1 root root 84709 Jan 2 2014 _mysql.c
drwxr-xr-x. 3 root root 4096 Nov 16 18:50 MySQLdb
-rw-rw-rw-. 1 root root 2352 Oct 8 2012 _mysql_exceptions.py
drwxr-xr-x. 2 root root 4096 Nov 16 01:41 MySQL_python.egg-info
-rw-rw-rw-. 1 root root 1689 Jan 2 2014 PKG-INFO
-rw-rw-rw-. 1 root root 3203 Oct 8 2012 pymemcompat.h
-rw-rw-rw-. 1 root root 1266 Oct 4 2012 README.md
-rw-rw-rw-. 1 root root 398 Jan 2 2014 setup.cfg
-rw-rw-rw-. 1 root root 1088 Oct 8 2012 setup_common.py
-rw-r--r--. 1 root root 1682 Nov 16 01:43 setup_common.pyc
-rw-rw-rw-. 1 root root 3392 Jan 2 2014 setup_posix.py
-rw-r--r--. 1 root root 3274 Nov 16 01:43 setup_posix.pyc
-rw-rw-rw-. 1 root root 609 Jan 2 2014 setup.py
-rw-rw-rw-. 1 root root 1334 Oct 8 2012 setup_windows.py
-rw-rw-rw-. 1 root root 664 Nov 16 01:57 site.cfg
drwxr-xr-x. 2 root root 4096 Nov 16 01:41 tests
[root@bogon MySQL-python-1.2.5]# python setup.py install
sh: /usr/local/bin/mysql_config: No such file or directory
Traceback (most recent call last):
File "setup.py", line 17, in <module>
metadata, options = get_config()
File "/home/kangql/Desktop/MySQL-python-1.2.5/setup_posix.py", line 43, in get_config
libs = mysql_config("libs_r")
File "/home/kangql/Desktop/MySQL-python-1.2.5/setup_posix.py", line 25, in mysql_config
raise EnvironmentError("%s not found" % (mysql_config.path,))
EnvironmentError: /usr/local/bin/mysql_config not found
[root@bogon MySQL-python-1.2.5]#

3. 使用 rpm -qa|grep mysql查看mysql的安装情况

mysql-devel-5.5.16-1.el6_4.x86_64

mysql-client-5.5.16-1.el6_4.x86_64

mysql-server-5.5.16-1.el6_4.x86_64

4. 使用rpm -e mysql-devel-5.5.16-1.el6_4.x86_64移除已有的安装包

5. 再次使用yum install mysql-devel 发现和mysql-server， mysql-client的版本有冲突

6. 去网上寻找相同的版本下载http://downloads.skysql.com/archive/index/p/mysql/v/5.5.16，下载64bit的rpm文件，使用rpm -ivh MySQL-devel-5.5.16-1.linux2.6.x86_64.rpm安装

7. 安装成功后，去/usr/bin/目录下，发现生成了mysql_config文件，继续pip install mysql-python，顺利完成

[root@bogon MySQL-python-1.2.5]# rpm -qa|grep mysql
mysql-libs-5.1.71-1.el6.x86_64
[root@bogon MySQL-python-1.2.5]# rpm -e mysql-libs-5.1.71-1.el6.x86_64
error: Failed dependencies:
libmysqlclient.so.16()(64bit) is needed by (installed) postfix-2:2.6.6-2.2.el6_1.x86_64
libmysqlclient.so.16(libmysqlclient_16)(64bit) is needed by (installed) postfix-2:2.6.6-2.2.el6_1.x86_64
mysql-libs is needed by (installed) postfix-2:2.6.6-2.2.el6_1.x86_64
[root@bogon MySQL-python-1.2.5]#

无法卸载，使用其他命令：

[root@bogon MySQL-python-1.2.5]# rpm -e --nodeps mysql-libs-5.1.71-1.el6.x86_64
[root@bogon MySQL-python-1.2.5]# rpm -qa|grep mysql
[root@bogon MySQL-python-1.2.5]#

yum 安装MySQL

redhat系统需要注册才能更新yum,删除yum更新，结果yum无法使用。。。。。。。

查找各种资料，发现yum在Python2.7下面无法使用

奶奶的，花费了一个下午才解决：

解决方法如下（网上其他不靠谱的方法就不说了）：

[root@Linux yum.repos.d]# yum
-bash: /usr/bin/yum: 没有那个文件或目录

[root@Linux yum.repos.d]# rpm -ivh --nodeps http://mirrors.163.com/centos/5/os/x86_64/CentOS//yum-fastestmirror-1.1.16-21.el5.centos.noarch.rpm
Retrieving http://mirrors.163.com/centos/5/os/x86_64/CentOS//yum-fastestmirror-1.1.16-21.el5.centos.noarch.rpm
Preparing... ########################################### [100%]

1:yum-fastestmirror ########################################### [100%]

[root@Linux yum.repos.d]# rpm -ivh --nodeps http://mirrors.163.com/centos/5/os/x86_64/CentOS/yum-fastestmirror-1.1.16-21.el5.centos.noarch.rpm
Retrieving http://mirrors.163.com/centos/5/os/x86_64/CentOS/yum-fastestmirror-1.1.16-21.el5.centos.noarch.rpm
Preparing... ########################################### [100%]

package yum-fastestmirror-1.1.16-21.el5.centos.noarch is already installed

[root@Linux yum.repos.d]# rpm -ivh --nodeps http://mirrors.163.com/centos/5/os/x86_64/CentOS/yum-metadata-parser-1.1.2-4.el5.x86_64.rpm
Retrieving http://mirrors.163.com/centos/5/os/x86_64/CentOS/yum-metadata-parser-1.1.2-4.el5.x86_64.rpm
Preparing... ########################################### [100%]

1:yum-metadata-parser ########################################### [100%]

[root@Linux yum.repos.d]# rpm -ivh --nodeps http://mirrors.163.com/centos/5/os/x86_64/CentOS/yum-3.2.22-40.el5.centos.noarch.rpm
Retrieving http://mirrors.163.com/centos/5/os/x86_64/CentOS/yum-3.2.22-40.el5.centos.noarch.rpm
Preparing... ########################################### [100%]

1:yum ########################################### [100%]

[root@Linux yum.repos.d]# yum

已解决。

[root@bogon Desktop]# yum list
There was a problem importing one of the Python modules
required to run yum. The error leading to this problem was:

No module named yum

Please install a package which provides this module, or
verify that the module is installed correctly.

It's possible that the above module doesn't match the
current version of Python, which is:
2.7.12 (default, Oct 18 2016, 02:30:08)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)]

If you cannot solve this problem yourself, please go to
the yum faq at:
http://wiki.linux.duke.edu/YumFaq

[root@bogon Desktop]# rpm -qa |grep yum
yum-fastestmirror-1.1.16-21.el5.centos.noarch
yum-3.2.22-40.el5.centos.noarch
yum-metadata-parser-1.1.2-4.el5.x86_64
[root@bogon Desktop]# whereis python
python: /usr/bin/python2.6-config /usr/bin/python2.6 /usr/bin/python /usr/lib/python2.6 /usr/lib/python2.4 /usr/lib64/python2.6 /usr/lib64/python2.4 /usr/include/python2.6 /usr/share/man/man1/python.1.gz
[root@bogon Desktop]# which yum
/usr/bin/yum
[root@bogon Desktop]# vi /usr/bin/yum
[root@bogon Desktop]#

[root@develop bin]# yum
There was a problem importing one of the Python modules
required to run yum. The error leading to this problem was:
No module named yum
Please install a package which provides this module, or
verify that the module is installed correctly.
It's possible that the above module doesn't match the
current version of Python, which is:
2.6.1 (r261:67515, Aug 7 2010, 11:36:17)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-44)]
If you cannot solve this problem yourself, please go to
the yum faq at:
http://wiki.linux.duke.edu/YumFaq
提示当前版本是2.6.1可能版本不匹配。没有yum依赖的包。
3、查看yum版本
[root@develop local]# rpm -qa |grep yum
yum-3.2.8-9.el5.centos.1
yum-metadata-parser-1.1.2-2.el5
4、查看python版本
[root@develop local]# whereis python
python: /usr/bin/python2.4 /usr/bin/python /usr/lib/python2.4 /usr/local/bin/python2.6 /usr/local/bin/python2.6-config /usr/local/bin/python /usr/local/lib/python2.6 /usr/share/man/man1/python.1.gz
果然装了两个版本python
5、执行python，查看到使用2.6.1的版本
[root@develop local]# python
Python 2.6.1 (r261:67515, Aug 7 2010, 11:36:17)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-44)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
6、猜测yum调用了高版本的python。
7、解决方法：
查找yum文件,并编辑此py文件
[root@develop local]# which yum
/usr/bin/yum
[root@develop local]# vi /usr/bin/yum

将
#!/usr/bin/python
改为:
#!/usr/bin/python2.4

然后保存OK.

补充：yum基于python写的。

发现无论怎么修改，yum还是无法使用。

[root@bogon Desktop]# python
Python 2.7.12 (default, Oct 18 2016, 02:30:08)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import yum
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named yum
>>>

[root@bogon Desktop]# rpm -qa |grep yum
yum-fastestmirror-1.1.16-21.el5.centos.noarch
yum-3.2.22-40.el5.centos.noarch
yum-metadata-parser-1.1.2-4.el5.x86_64
[root@bogon Desktop]# whereis python
python: /usr/bin/python2.6-config /usr/bin/python2.6 /usr/bin/python /usr/lib/python2.3 /usr/lib/python2.6 /usr/lib/python2.4 /usr/lib64/python2.3 /usr/lib64/python2.6 /usr/lib64/python2.4 /usr/include/python2.6 /usr/share/man/man1/python.1.gz
[root@bogon Desktop]#

查看系统使用的python版本

ls /usr/bin |grep python

修改yum文件
#vi /usr/bin/yum
将 #!/usr/bin/python 修改为 #!/usr/bin/python2.6

如果有大神知道怎么处理，请跟我联系或者留言~

ps:yum的安装程序链接http://mirrors.163.com/centos/6/os/x86_64/Packages/

MySQL下载地址：http://dev.mysql.com/downloads/file/?id=466135

MySQL数据库安装好后保存数据，报错误信息：[scrapy] ERROR: (1366, "Incorrect string value: '\\xE4\\xB8\\xAD' for column 'productName' at row 1 原因是数据库字段和Python字段的类型不一致，将数据库字段的类型改为utf-8。参照：http://www.lai18.com/content/9984292.html

另外，爬取出来的数据有[。。。。]这样的方括号，在语句末尾添加[0]，这样可以去掉方括号。比如：

sel.xpath('//span[contains(@id,"lblTimeLimit")]/descendant::text()').extract()[0]

Java不是C

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
scrapy 安装数据库mysql错误汇总和操作步骤

操作步骤：　　1、检查python是否支持mysql[root@bogon ~]# pythonPython 2.7.10 (default, Jun 5 2015, 17:56:24) [GCC 4.4.4 20100726 (Red Hat 4.4.4-13)] on linux2Type "help", "copyright", "credits" or "license
复制链接

扫一扫

专栏目录