在scrapy_splash中加载本地Cookies

Splash→Scrapy：

首先，lua脚本中要返回Cookies：

return {cookies = splash:get_cookies()}

然后，在spider中调用response.cookiejar即可得到返回的Cookies

Scrapy→Splash

把Cookies作为参数传递到SplashRequest()中：

def start_requests(self):
        local = "D:\\...\\VertifiedCookies.txt"
        with open(local, 'r+') as f:
            cookies = eval(f.read())
        url = 'https://www.example.com'
        script = """
        function main(splash)
            splash:clear_cookies()
            splash:init_cookies(splash.args.cookies)
            assert(splash:go(splash.args.url))
            assert(splash:wait(splash.args.wait))
            return {cookies = splash:get_cookies(),html = splash:html()}
        end
        """
        yield SplashRequest(url, self.parse, endpoint='execute', args={'lua_source':script,'wait':1}, cookies=cookies)

传入数据的参数类型与scrapy.Request中的cookies相同：

转载于:https://www.cnblogs.com/lokvahkoor/p/10805364.html