scrapy 中遇到的问题

最新推荐文章于 2024-04-24 16:55:30 发布

山大王110

最新推荐文章于 2024-04-24 16:55:30 发布

阅读量3.7k

点赞数

分类专栏：爬虫

本文链接：https://blog.csdn.net/u013155359/article/details/82149932

版权

爬虫专栏收录该内容

9 篇文章 0 订阅

订阅专栏

2018-08-28 16:09:32 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (302) to <GET http://bj.ganji.com/fang1/3558462755x.htm?adtype=3> from <GET http://jing.58.com/adJump?adType=3&target=pZwY0jCfsvFJshI6
UhGGshPfUiqhmyOMnit1PHN3PjmzP1NdXaOCIAY_uadBnvuhujmQnBYvPhndsHwBm1mVmhNkriY1PHKBPym1Pynzryc_nHN1PHEYnH0LPHbQPHDksakhUAqMuv-8gLR1ugFxpyEqnaubpgPkgvP6IANqPzkznaubpgPkgvQfmv7_5HDzsjDLPauWUA--UMwxIgP-0-qGujYhuyOYpgwOp
yEqn1NdrjEvnW0dPiud0vRzpyEqP101rHmknHDQFh-VuybqFhR8IA-YXgwO0ANqPau-UMwGIZ-xmv7YuHYhuyOYpgwOgvQfmv7_5iubpgPkgLwGUyNqnHN1PHEYn10vrjDQraukULPGIA-fUWYzFhP_pyPopyEqmHEduAnduAEVnyFWnaYYnWcLsH--nHbVrHN1mWcvnjEOrjnLFMK60h
7V5iukUA7YuhqzUHYVniu_pgPYXhEqUA-1IZGbgvIJFMK6uvRxIZ-kuHYQsjD1FMKzUvwdmLwGujYQnjTQn9&end=end>
2018-08-28 16:09:32 [scrapy.core.engine] DEBUG: Crawled (200) <GET http://bj.ganji.com/fang1/3558465548x.htm?adtype=3> (referer: http://bj.ganji.com/fang1/chaoyang/o1/)

1.scrapy爬虫时遇到的重定向问题：

DEBUG:Redirecting(302) to <GET>类问题

产生原因：页面被跳转到登录页面了，触动了赶集网的防爬装置。

解决方法：

2.DEBUG: Crawled (200) <GET >类问题