从源码开始学习Scrapy系列13-startproject指令

版权声明:本文为博主原创文章,支持转载,但希望你加上转载地址,原创不易,且转且珍惜。 https://blog.csdn.net/wang1472jian1110/article/details/80406799

前言

startproject指令用于快速创建scrapy项目

代码调试

获取项目名,项目目录

project_name = args[0]
project_dir = args[0]

判定当前项目目录是否存在scrapy.cfg文件

if exists(join(project_dir, 'scrapy.cfg')):
    self.exitcode = 1
    print('Error: scrapy.cfg already exists in %s' % abspath(project_dir))
    return

项目名正确性检验

if not self._is_valid_name(project_name):
    self.exitcode = 1
    return

将模板目录templates下的内容拷贝到当前项目目录下

self._copytree(self.templates_dir, abspath(project_dir))

模板目录

def templates_dir(self):
    _templates_base_dir = self.settings['TEMPLATES_DIR'] or \
        join(scrapy.__path__[0], 'templates')
    return join(_templates_base_dir, 'project')

将module替换成项目名

move(join(project_dir, 'module'), join(project_dir, project_name))

选择spider模板文件,并填充内容,生成代码文件

for paths in TEMPLATES_TO_RENDER:
    path = join(*paths)
    tplfile = join(project_dir,
        string.Template(path).substitute(project_name=project_name))
    render_templatefile(tplfile, project_name=project_name,
        ProjectName=string_camelcase(project_name))
print("New Scrapy project %r, using template directory %r, created in:" % \
      (project_name, self.templates_dir))
print("    %s\n" % abspath(project_dir))
print("You can start your first spider with:")
print("    cd %s" % project_dir)
print("    scrapy genspider example example.com")


github地址

https://github.com/wangrenlei/debug_scrapy


结束

展开阅读全文

没有更多推荐了,返回首页