scrapy python版本_Python和Scrapy：Scrapy版本的问题

I'm using Ubuntu 14.04 x64, I've followed the Scrapy docs to install the package using pip:

pip install scrapy

Then I followed the initialization of the example project and try to execute the example spider:

scrapy crawl example

and I get this error:

2015-02-23 10:23:42+0100 [scrapy] INFO: Scrapy 0.14.4 started (bot: example)

2015-02-23 10:23:42+0100 [scrapy] DEBUG: Enabled extensions: LogStats, TelnetConsole, CloseSpider, WebService, CoreStats, MemoryUsage, SpiderState

Traceback (most recent call last):

File "/usr/local/bin/scrapy", line 11, in

sys.exit(execute())

File "/usr/lib/python2.7/dist-packages/scrapy/cmdline.py", line 132, in execute

_run_print_help(parser, _run_command, cmd, args, opts)

File "/usr/lib/python2.7/dist-packages/scrapy/cmdline.py", line 97, in _run_print_help

func(*a, **kw)

File "/usr/lib/python2.7/dist-packages/scrapy/cmdline.py", line 139, in _run_command

cmd.run(args, opts)

File "/usr/lib/python2.7/dist-packages/scrapy/commands/crawl.py", line 43, in run

spider = self.crawler.spiders.create(spname, **opts.spargs)

File "/usr/lib/python2.7/dist-packages/scrapy/command.py", line 34, in crawler

self._crawler.configure()

File "/usr/lib/python2.7/dist-packages/scrapy/crawler.py", line 36, in configure

self.spiders = spman_cls.from_crawler(self)

File "/usr/lib/python2.7/dist-packages/scrapy/spidermanager.py", line 37, in from_crawler

return cls.from_settings(crawler.settings)

File "/usr/lib/python2.7/dist-packages/scrapy/spidermanager.py", line 33, in from_settings

return cls(settings.getlist('SPIDER_MODULES'))

File "/usr/lib/python2.7/dist-packages/scrapy/spidermanager.py", line 23, in __init__

for module in walk_modules(name):

File "/usr/lib/python2.7/dist-packages/scrapy/utils/misc.py", line 65, in walk_modules

submod = __import__(fullpath, {}, {}, [''])

File "/home/alvaro/vagrantenvs/example/example/spiders/example_spider.py", line 3, in

class ExampleSpider(scrapy.Spider):

AttributeError: 'module' object has no attribute 'Spider'

I checked with pip freeze | grep 'Scrapy' to check the installed version of Scrapy and I got:

Scrapy==0.24.4

but as you can see in the traceback error:

2015-02-23 10:23:42+0100 [scrapy] INFO: Scrapy 0.14.4 started (bot: example)

I tried to uninstall and reinstall

I tried also to update pip, uninstall scrapy and reinstall

I tried to install it using apt-get

Why If I installed the latest version (0.24.4) it seems it's executing 0.14.4 ?

Is this issue related with Scrapy, Pip or both ?

Edit:

Here it is the file example_spider.py:

import scrapy

class ExampleSpider(scrapy.Spider):

name = "example"

allowed_domains = ["dmoz.org"]

start_urls = [

"http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",

"http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"

]

def parse(self, response):

filename = response.url.split("/")[-2]

with open(filename, 'wb') as f:

f.write(response.body)

If I execute scrapy version as @aberna sugested I got:

Scrapy 0.14.4

解决方案

I had a similar issue back then and It was because the Ubuntu package was messing with the pip package.

When I installed Scrapy using pip and try to run it, scrapy says that the package was missing some dependencies and then I installed Scrapy using apt-get and there is where I screwed up with the package version.

I recommend you to remove both packages from Ubuntu and Pip and then reinstall only using pip:

apt-get remove --purge python-scrapy

pip uninstall scrapy

After those actions finished you can install it again using pip and then if you do scrapy version you should get as response the newest version, in your case it should be 0.24.4.

EDIT

You can read this in Scrapy Documentation:

python-scrapy is a different package provided by official debian repositories, it’s very outdated and it isn’t supported by Scrapy team.

To install Scrapy in Ubuntu they say:

Import the GPG key used to sign Scrapy packages into APT keyring:

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 627220E7

Create /etc/apt/sources.list.d/scrapy.list file using the

following command:

echo 'deb http://archive.scrapy.org/ubuntu scrapy main' | sudo tee /etc/apt/sources.list.d/scrapy.list

3.Update package lists and install the scrapy-0.24 package:

sudo apt-get update && sudo apt-get install scrapy-0.24