PHP如何启动scrapy,php教程博客

Scrapy简介

Scrapy是一个快速,高效的网页抓取python框架。主要用于Web抓取&提取信息&格式化数据。经常用此做数据挖掘、检测、测试等。

安装所需软件

python

python

python

安装步骤

1、安装Python

官网下载python(http://www.python.org/ftp/python/2.7.5/python-2.7.5.msi)& 傻瓜安装将python路径(D:\Python27;D:\Python27\Scripts;)加入环境变量

验证是否安装okC:\Users\admin>python

Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win

32

Type "help", "copyright", "credits" or "license" for more information.

>>>

2、安装setuptools

官网下载setuptools(http://pypi.python.org/pypi/setuptools)& 傻瓜安装

3、安装Zope.Interface

官网下载Zope.Interface(http://pypi.python.org/pypi/zope.interface/)& 傻瓜安装验证是否安装okC:\Users\admin>python

Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win

32

Type "help", "copyright", "credits" or "license" for more information.

>>> import zope.interface

>>>

4、安装Twisted

官网下载Twisted(http://twistedmatrix.com/trac/wiki/Downloads)& 傻瓜安装

5、安装w3lib

官网下载w3lib(http://pypi.python.org/pypi/w3lib)安装

#进入插件目录并执行命令安装

>D:\python-plugin\w3lib-1.3>python setup.py install

验证

D:\python-plugin\w3lib-1.3>python

Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win

32

Type "help", "copyright", "credits" or "license" for more information.

>>> import w3lib

>>>

6、安装libxml2

官网下载libxml2(http://users.skynet.be/sbi/libxml-python/)& 傻瓜安装

7、安装pyOpenSSL

官网下载pyOpenSSL(https://pypi.python.org/pypi/pyOpenSSL)& 傻瓜安装

8、安装scrapy

官网下载scrapy(https://pypi.python.org/pypi/Scrapy)安装

#进入scrapy目录并执行安装

>D:\python-plugin\Scrapy-0.16.5>python setup.py install

验证

D:\python-plugin\Scrapy-0.16.5>scrapy

Scrapy 0.16.5 - no active project

Usage:

scrapy [options] [args]

Available commands:

fetch Fetch a URL using the Scrapy downloader

runspider Run a self-contained spider (without creating a project)

settings Get settings values

shell Interactive scraping console

startproject Create new project

version Print Scrapy version

view Open URL in browser, as seen by Scrapy

[ more ] More commands available when run from project directory

Use "scrapy -h" to see more info about a command

D:\python-plugin\Scrapy-0.16.5>

安装完毕 OK