目的:为了获得二手房相关信息(包括小区,地址,户型,年份,价格等)
1、获取网页数据
import requests from lxml import html etree = html.etree headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36'} url = 'https://hf.lianjia.com/ershoufang/' page_text = requests.get(url=url,headers=headers).text
2、数据解析
tree = etree.HTML(page_text) li_list = tree.xpath('//ul[@class="sellListContent"]/li')
3、用xpath解析出详细信息
for li in li_list : title = li.xpath('.//div[@class="title"]/a/text()') houseIcon = li.xpath('.//div[@class="houseInfo"]/text()') target = li.xpath('.//div[@class="positionInfo"]/a/text()') money = li.xpath('.//div[@class="totalPrice total