学习笔记(08):零基础搞定Python数据分析与挖掘-爬虫案例2-- 链家二手房

最新推荐文章于 2024-08-13 21:38:27 发布

Glen_qiu

最新推荐文章于 2024-08-13 21:38:27 发布

阅读量216

点赞数

分类专栏：研发管理文章标签：数据处理数据管理 RFM模型线性回归案例其他

本文链接：https://blog.csdn.net/Glen_qiu/article/details/105621564

版权

研发管理专栏收录该内容

8 篇文章 0 订阅

订阅专栏

这篇博客介绍了如何使用Python进行链家二手房数据的爬虫实战，通过requests和BeautifulSoup库抓取房源信息，包括区域、房屋详情、面积、朝向、装修、楼层、年份、总价和单价等数据，并将结果整理成DataFrame以便后续的数据分析和挖掘。

摘要由CSDN通过智能技术生成

立即学习:https://edu.csdn.net/course/play/6861/341445?utm_source=blogtoedu

爬虫抓取练习 -房产信息抓取

import requests
import bs4
import pandas as pd
# 链家网
url = r'https://sh.lianjia.com/ershoufang/pudong/pg1/'
#heads = ｛'user-agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.113 Safari/537.36'｝
#搜房网
#url = r'https://wuhan.esf.fang.com/'
#response = requests.get(url,headers = heads)
response = requests.get(url)
soup = bs4.BeautifulSoup(response.text)
soup.findAll(name = 'a',attrs ={'data-el':'region'})
name = [i.text.strip() for i in soup.findAll(name = 'a',attrs ={'data-el':'region'})]
housInfo = [i.text.split('|')[0].strip() for i in soup.findAll(name = 'div',attrs ={'class':'houseInfo'})]
size = [float(i.text.split('|')[1].strip()[:-2]) for i in soup.findAll(name = 'div',attrs ={'class':'houseInfo'})]
direction = [i.text.split('|')[2].strip() for i in soup.findAll(name = 'div',attrs ={'class':'houseInfo'})]
zhuangxiu = [i.text.split('|')[3].strip() for i in soup.findAll(name = 'div',attrs ={'class':'houseInfo'})]
fool = [i.text.split('|')[4].strip() for i in soup.findAll(name = 'div',attrs ={'class':'houseInfo'})]
year = [i.text.split('|')[5].strip() for i in soup.findAll(name = 'div',attrs ={'class':'houseInfo'})]
totalPrice = [float(i.text[:-1]) for i in soup.findAll(name = 'div',attrs ={'class':'totalPrice'})]
unitPrice = [int(i.text[2:-4]) for i in soup.findAll(name = 'div',attrs ={'class':'unitPrice'})]
pd.DataFrame({'name':name,'housInfo':housInfo,'size':size,'direction':direction,'zhuangxiu':zhuangxiu,'fool':fool,'year':year,'totalPrice':totalPrice,'unitPrice':unitPrice})