爬虫之成都单片机实习岗

爬虫之成都单片机实习岗

本来计划使用scrapy,结果呢由于技术不精，最终跟流水账似的代码，还好得到的结果还不错。爬取了boss zhipin网站的数据，发现爬取一次就更新一次cookie，不然服务器返回一段不认识的html。收集了职位，发布时间，薪资地点，链接等信息存在excel里面。
存储数据用的是openxl模块，因为不常用，在此记录使用方法。
导入
from openxl import Workbook
新建工作簿
wb = Workbook()
选择默认工作表
sheet = wb.active
给工作表重命名
sheet.title = ‘考勤统计表’
写入多行数据
for row in data:
sheet.append(row)
保存excel文件
wb.save(‘考勤统计.xlsx’)

import requests
from bs4 import BeautifulSoup
import time
# 从 openpyxl 引入 Workbook（工作簿）类
from openpyxl import Workbook
from concurrent import futures
#为了判断两次登录的cookie是否改变,设置lstr和newstr,结果为false
lstr='''JSESSIONID=""; _bl_uid=d1kUIhkplabhvppR98Uyub0tmIUt; lastCity=100010000; __fid=0e1350eb0a068bdf26e42673e0313c36; __g=-; Hm_lvt_194df3105ad7148dcf2b98a91b5e727a=1605588068,1605599043; __zp_stoken__=4eeebODFXbSV8Uyw1LhtJejd%2FSSFCTF42a1ZBLD8yZBMHT0ldO0xTenB9JTdOJl0VfW5uVT1NXEIJDV4XKWdHOAl%2FJx4WHBEQWQ9vD21DDERHNRwYVnQ2PW0GRy1PWiRiez9kRnhELUR2bwl0JQ%3D%3D; toUrl=https%3A%2F%2Fwww.zhipin.com%2F; __c=1605588067; __l=r=https%3A%2F%2Fopen.weixin.qq.com%2F&l=%2Fwww.zhipin.com%2Fwechat%2Faccount%2Fbind%3Fwct%3D8a4295fcae3d15e1iBILvau9dyNTxuPjAofwQ4fcwaKRbWggp98M8w~~&g=&friend_source=0&friend_source=0; __a=34580553.1605588067..1605588067.82.1.82.82; Hm_lpvt_194df3105ad7148dcf2b98a91b5e727a=1605602481'''
headers={
   'user-agent':"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36",
         'cookie':'''JSESSIONID=""; _bl_uid=d1kUIhkplabhvppR98Uyub0tmIUt; lastCity=100010000; __fid=0e1350eb0a068bdf26e42673e0313c36; __g=-; Hm_lvt_194df3105ad7148dcf2b98a91b5e727a=1605588068,1605599043; __zp_stoken__=4eeebOHxpOioDQQNqVF9HH04%2Fb3pBeiJYT1YxXVcAdFlCQHdtaHZIPwp6IyJ7LXIQfW5uVT1NdjAcTBcXLg4ZMzRnJRhgPmRcDQZqYwAxDERHNRwYU1s9CHgAQFcKQR4xez9kRnhELUR2bwl0JQ%3D%3