neo的博客

python

ok，走起，我们先把网址贴出来 要爬的网址是：https://movie.douban.com/cinema/nowplaying/chengdu/先把这个网页先爬下来import urllib2resq = urllib2.Request("https://movie.douban.com/cinema/nowplaying/chengdu/") response = urllib2.urlope

改下代码，观察了下，每页有20个评论，那我们就每次让page自增20，一直循环爬下去，先试试效果吧，小伙伴们increase = 0 result = [] while 1: page = 0 + increase increase += 20 try: url = 'https://movie.douban.com/subject/'+movie_id+'/

#coding:utf8import urllib2 import json import xlwt import oswbk = xlwt.Workbook() sheet = wbk.add_sheet(u'新华网旅游相关信息') sheet.write(0, 0, u'标题') sheet.write(0, 1, u'url地址') title_list = [] url_list = []

先发下目录架构get_url:#-*- coding:utf8 -*- import requests def get_url(): result = requests.get(url, headers=config.headers) resp = result.content _data = json.loads(resp) pattern = re.comp...