接上一篇,我们依据第一页的链接爬取了第一页的博客,我们不难发现,每一页的链接就仅仅有一处不同(页码序号),我们仅仅要在上一篇的代码外面加一个循环,这样就能够爬取全部博客分页的博文。也就是全部博文了。
# -*- coding : -utf-8 -*-
import urllib
import time
url = [' ']*350
page = 1
link = 1
while page <=7://眼下共同拥有7页。3
con = urllib.urlopen('http://blog.sina.com.cn/s/articlelist_1191258123_0_'+str(page)+'.html').read()
i = 0
while title != -1 and href != -1 and html != -1 and i<350:
url[i] = con[href + 6:html + 5]
content = urllib.urlopen(url[i]).read()
open(r'allboke/'+url[i][-26:],'w+').write(content)