利用python网络爬虫爬取赶集网数据

最新推荐文章于 2021-03-27 20:56:27 发布

liforpython

最新推荐文章于 2021-03-27 20:56:27 发布

阅读量3.2k

点赞数

分类专栏： python爬虫文章标签： python 网络爬虫数据赶集网

本文链接：https://blog.csdn.net/liforpython/article/details/75453654

版权

python爬虫专栏收录该内容

2 篇文章 0 订阅

订阅专栏

import csv
import requests
import re
headers = {'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36'}
url = 'http://sh.ganji.com/zpbanyungong/o1/'
with open('E:/infomation.csv', 'wt', encoding='utf8',newline='') as f:
    csv_writer = csv.writer(f, dialect='excel')#这里打开csv文件开始写入  
    for i in range(1,2):
        new_url = 'http://sh.ganji.com/zpbanyungong/o' + str(i) +'/'
        content = requests.get(new_url,headers=headers).text
        pattern = re.compile('gjalog="100000002704000200000010@atype=click">(.*?).*?
   
   
    
    (.*?)
   
   .*?
   
   
    
    (.*?)
   
   ',re.S)
        results = re.findall(pattern,content)
        for result in results:
            name,money,contrict = result
            name = re.sub('/s+','',name).strip()
            money = re.sub('/s+','',money).strip()
            contrict = re.sub('/s+','',contrict).strip()
            csv_writer.writerow([name, money, contrict])  #
            print(name,money,contrict)

以上就是本文的代码啦