____

经过十几万网页采集测试，有效率99%def pick_charset(html): """ 从文本中提取 meta charset :param html: :return: """ charset = None m = re.compile('<meta .*(http-equiv="?Content-Type"?.*)?charset="?

实战十几万网页采集后的修正版：python彻底解决网页采集乱码问题def download_page(indexurl, timeout=10): """ 下载网站数据并返回 :param indexurl: :param timeout: :return: """ try: res = urllib2.urlopen(inde