python爬虫“糗事百科”

原文链接地址：http://cuiqingcai.com/990.html，由于糗百改动，上述代码的正则表达式需改变，同时增加了匹配附图部分以及去除html中的换行符，代码如下:

# -*- coding:utf-8 -*-
import urllib
import urllib2
import re

url = 'http://www.qiushibaike.com/8hr/page/3/?s=4937555'
user_agent = 'Mozilla/5.0 (Windows NT 6.1; WOW64)'
headers = {'User-Agent' : user_agent}
try:
    request = urllib2.Request(url,headers = headers)
    response = urllib2.urlopen(request)
    content = response.read().decode('utf-8')
    #print content
    pattern = re.compile(r'.*?<h2>(.*?)</h2>.*?<div class="content">.*?<span>(.*?)</span>.*?</div>.*?</a>(.*?)<div class="stats"'+
        r'>.*?<i class="number">(.*?)</i>',re.S)
    items = re.findall(pattern,content)
    for item in items:
        pattern1 = re.compile(r'<br/>')
        context = re.sub(pattern1,r'',item[1])
        haveImg = re.search(r'pic',item[2])
        if haveImg:
            pattern2 = re.compile(r'.*?<img src="(.*?)".*?>.*?',re.S)
            items1 = re.findall(pattern2,item[2])
            print item[0],context,items1[0],item[3]
        else:
            print item[0],context,item[3]
except urllib2.URLError, e:
    if  hasattr(e,"code"):
        print e.code
    if  hasattr(e,"reason"):
        print e.reason

另附可用代码，但没有去除含有附图的item：http://www.cnblogs.com/farewell-farewell/p/6046337.html