使用python抓取CSDN关注人的所有发布的文章

最新推荐文章于 2014-09-09 17:27:18 发布

jiangfullll

最新推荐文章于 2014-09-09 17:27:18 发布

阅读量1k

点赞数

分类专栏： python 文章标签： python爬虫抓取数据

本文链接：https://blog.csdn.net/startupmount/article/details/30060575

版权

python 专栏收录该内容

21 篇文章 0 订阅

订阅专栏

# -*- coding: utf-8 -*-
"""
@author: jiangfuqiang
"""
import re
import urllib2
import cookielib
import time

def startParser(author,page=1):
    reg = r'<a href="/\w+/article/details/\d+">\s*\t*\n*\s*\t*\s*.*?\t*\n*\t*\s*</a>'
    cj = cookielib.LWPCookieJar()
    cookie_support = urllib2.HTTPCookieProcessor(cj)
    opener = urllib2.build_opener(cookie_support,urllib2.HTTPHandler)
    urllib2.install_opener(opener)
    headers = {'User-Agent' : 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:14.0) Gecko/20100101 Firefox/14.0.1',
           'Referer' : '    http://my.csdn.net/my/favorite'}
    flag = True
    while flag == True:
        time.sleep(2)
        url = "http://blog.csdn.net/%s/article/list/%d"%(author,page)
        req = urllib2.Request(url,headers=headers)
        resp = urllib2.urlopen(req)
        data = resp.read()
        regex = re.compile(reg,flags=re.MULTILINE)
        result = regex.findall(data)

        for rd in result:
            print rd
        if len(result) < 20:
            flag = False
        page = page + 1
    print 'success............page:%d'%page
    #print result.group()

if __name__ == '__main__':

    startParser('yiyaaixuexi',1)

这篇 python抓取收藏的文章链接和标题中有python发送邮件的代码，可以将此程序稍微改动之后将文章链接发送的邮箱以便以后查阅