scrapy_redis的设置

spiders的xxxx.py中在导入此包
from scrapy_redis.spiders import RedisSpider

修改如下代码设置redis的键
#删除start_urls
#start_urls = ['http://python.jobbole.com/all-posts/']
redis_key = 'lhy:rediskey'

在settings文件中加入

SCHEDULER = "scrapy_redis.scheduler.Scheduler"
DUPEFILTER_CLASS = "scrapy_redis.dupefilter.RFPDupeFilter"
SCHEDULER_PERSIST = True

REDIS_HOST = '127.0.0.1'
REDIS_PORT = 6379
REDIS_PASSWORD = 123456

如果需要让其他人也访问你的mysql的话需要设置mysql

进入mysql 

select user,host from mysql.user;查看

grant all on *.* to root@'%' identified by '123456'

flush privileges;