深圳数据分析岗需求分析

深圳数据分析岗位招聘分析

此次案例只做为练习用

目的：爬取boss直聘网站中深圳地区的数据分析岗位数据，进行简单的数据分析练习

问题：

1.各个区对数据分析岗位的需求
2.主要是哪些行业在招聘
3.招聘的要求及薪资
3.1学历与工作经验要求及薪资情况
3.2不同行业的薪资分布
3.3不同公司规模的薪资分布

步骤：

1.数据的获取
2.数据的处理
3.数据分析，数据展现
4.总结

内容：

数据说明：

pisitionId:公司链接
positionName：岗位名称
salary_min：最低薪资
salary_max：最高薪资
city：区域
experience：经验	
education：学历
companyShortName：公司名称	
industryField：行业
financeStage：融资情况
companySize：公司规模

数据清洗：

1.将薪资区间处理成平均值，并用数字表示，
2.将city处理成区域，
3.公司规模去掉人及以上

数据分析

通过pandas，matplolit，seaborn进行分析及数据可视化

总结：

1. 各个区对数据分析岗位的需求

南山区对数据分析岗位的需求最大，占47.5%，其次是福田区占25%，罗湖区为10%，宝安区为9.17%，龙岗区为6.67%，龙华区为1.67%

2. 主要是哪些行业在招聘

发现有些行业的内容是有相关性的，这个定义比较宽泛，互联网行业对数据分析岗位的需求量是最大的达到50%，其次是电子商务，还有计算机软件，游戏等其他行业。

3. 招聘的要求及薪资

3.1 学历与工作经验要求及薪资情况

数据分析师对学历的要求不算太严格，本科学历的需求占比达到79.%，大专及不限学历的占比达到18.3%，但对工作经验的要求是比较高的，没有工作经验的占比只有10.8%.硕士的薪资明显高于其他学历，平均薪资有27500，大专的学历薪资较低，均值只有12000，工作经验越久，薪资水平也高，工作年限达到3年后，薪资会有一个大的提升,均值能到27000以上，1年以内和应届毕业生的薪资在6000左右。

3.2 不同行业的薪资分布

移动互联网和互联网行业的薪资水平是非常高的，均值能达到27000。电子商务和互联网金融相当低一些，平均薪资也能达到15000左右。

3.3不同公司规模的薪资分布

从结果上看，20-99人的公司给的月薪低，公司规模越大，能提供的薪资也多。

数据爬取

#爬取数据
from lxml import etree
import requests
import time
import csv

#发送网络请求，获取页面数据
def get_page(i):
    req_url =("https://www.zhipin.com/c101280600/?query=%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90%E5%B8%88&page={}&ka=page-{}".format(i,i))
    headers = {
   
    "cookie": "lastCity=101280600; _uab_collina=156196959854179320294632; __c=1569823345; JSESSIONID=""; sid=sem; toUrl=/; __g=sem; __l=l=%2Fwww.zhipin.com%2F%3Fsid%3Dsem_pz_bdpc_dasou_title&r=https%3A%2F%2Fsp0.baidu.com%2F9q9JcDHa2gU2pMbgoY3K%2Fadrc.php%3Ft%3D06KL00c00fDIFkY0luu-0KZEgs7VhG7I0000079yiNC00000XrRPGL.THdBULP1doZA80K85yF9pywd0Znqm1N9uWwbujRsnj01uhczn6Kd5Hc1PDnsnDwAPjfdfYc1wW7aPYujrRfkrHF7nRuAwRmv0ADqI1YhUyPGujY1nWb1nHmvrHD4FMKzUvwGujYkP6K-5y9YIZK1rBtEILILQMGCpgKGUB4WUvYE5LPGujd1uydxTZGxmhwsmdqbmgPEINqYpgw_ufKWThnqnHTzr0%26tpl%3Dtpl_11534_19968_16032%26l%3D1514755672%26attach%3Dlocation%253D%2526linkName%253D%2525E6%2525A0%252587%2525E5%252587%252586%2525E5%2525A4%2525B4%2525E9%252583%2525A8-%2525E6%2525A0%252587%2525E9%2525A2%252598-%2525E4%2525B8%2525BB%2525E6%2525A0%252587%2525E9%2525A2%252598%2526linkText%253DBOSS%2525E7%25259B%2525B4%2525E8%252581%252598%2525E2%252580%252594%2525E2%252580%252594%2525E6%252589%2525BE%2525E5%2525B7%2525A5%2525E4%2525BD%25259C%2525EF%2525BC%25258C%2525E6%252588%252591%2525E8%2525A6%252581%2525E8%2525B7%25259F%2525E8%252580%252581%2525E6%25259D%2525BF%2525E8%2525B0%252588%2525EF%2525BC%252581%2526xp%253Did(%252522m3293166919_canvas%252522)%25252FDIV%25255B1%25255D%25252FDIV%25255B1%25255D%25252FDIV%25255B1%25255D%25252FDIV%25255B1%25255D%25252FDIV%25255B1%25255D%25252FH2%25255B1%25255D%25252FA%25255B1%25255D%2526linkType%253D%2526checksum%253D136%26ie%3DUTF-8%26f%3D8%26tn%3Dbaidu%26wd%3Dboss%25E7%259B%25B4%25E8%2581%2598%26oq%3Dboss%25E7%259B%25B4%25E8%2581%2598%26rqlang%3Dcn&g=%2Fwww.zhipin.com%2Fuser%2Fsem7.html%3Fsid%3Dsem%26qudao%3Dbdpc_baidu-%25E5%258D%258E%25E5%2593%2581%25E5%258D%259A%25E7%259D%25BF02A18KA0679%26plan%3DNew-%25E5%2593%2581%25E7%2589%258C%25E8%25AF%258D-05%26unit%3D%25E5%2593%2581%25E7%2589%258C%252B%25E6%25B1%2582%25E8%2581%258C%25E6%2589%25BE%25E5%25B7%25A5%25E4%25BD%259C%26keyword%3Dboss%25E7%259B%25B4%25E8%2581%2598%25E8%2581%258C%25E4%25BD%258D%26bd_vid%3D9558004979476257735&friend_source=0&friend_source=0; Hm_lvt_194df3105ad7148dcf2b98a91b5e727a=1570418934,1570512475,1570512908,1570536201; __a=71773995.1561969598.1563265159.1569823345.116.3.100.3; Hm_lpvt_194df3105ad7148dcf2b98a91b5e727a=1570536209; __zp_stoken__=bcd9e%2BD2P33IFm6E1D3GlpNm%2B52drGHqvNaaT7%2FNK1L4EM2bzZdjHfKw5BfCBB0XOafy9hwjKTgMOtmI2NkMiMjqCg%3D%3D",
    "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3",
    "Host": "www.zhipin.com",
    "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36",
    "accept-encoding": "gzip, deflate, br",
    "accept-language": "zh-CN,zh;q=0.9",
    "Connection": "keep-alive"}
    
    response = requests.get(url=req_url,headers=headers)
    response = response.content.decode("utf-8")
    return response

#解析数据，获取当前页面的公司信息
def get_companys(response): 
    html = etree.HTML(response)
    company_path = "//div[@class='job-primary']"
    companys = html.xpath(company_path)
    return companys

#返回职位的详细信息，
def get_job(companys): 
        positionName = c.xpath(".//h3[@class='name']//div[@class='job-title']/text()")[0]
        salary = c.xpath(".//h3[@class='name']//span/text()")[0]
        city = c.xpath(".//p/text()")[0]
        experience = c.xpath(".//p/text()")[1]
        education = c.xpath(".//p/text()")[2]
        companyShortName = c.xpath(".//div[@class='info-company']//a/text()")[0]
        industryField = c.xpath(".//div[@class='info-company']//p/text()")[0]
        financeStage= c.xpath(".//div[@class='info-company']//p/text()")[1]
        companySize= c.xpath(".//div[@class='info-company']//p/text()")[-1]
        positionId = c.xpath(".//div[@class='info-primary']//a/@href")

        details = {