1. 程序创建第一步,需要设置这个在setting文件中
# Crawl responsibly by identifying yourself (and your website) on the user-agent
USER_AGENT = 'spiderTest1 (+https://jbk.39.net/)'
1. 程序创建第一步,需要设置这个在setting文件中
# Crawl responsibly by identifying yourself (and your website) on the user-agent
USER_AGENT = 'spiderTest1 (+https://jbk.39.net/)'