1.代码可以直接运行,请下载anaconda并安装,用spyder方便查看变量
或者可以查看生成的excel文件
2.依赖库,命令行运行(WIN10打开命令行快捷键:windows+x组合键,然后按a键):
pip install BeautifulSoup4
pip install requests
3.爬取的网站是安居客(厦门)网站,可以进入https://xm.fang.anjuke.com/loupan/all/进行观察
4.关于如何判断代码是python2还是python3,print('')为python3,print ''为python2
# -*- coding: utf-8 -*-
"""
Created on Sun Jan 14 19:07:39 2018
@author: Steven Lei
"""
def getHousesDetails(url):
import requests
from bs4 import BeautifulSoup
request = requests.get(url)
request.encoding = 'utf-8'
soup = BeautifulSoup(request.text,'lxml')
houses = soup.select('.item-mod')[3:]
housesDetails = []
for house in houses:
#获取楼盘名字
houseName = house.select('.items-name