Python爬虫入门:链家二手房房价,位置爬取

Python爬虫入门:链家二手房房价,位置爬取

# -*- coding: utf-8 -*-
"""
Created on Wed Sep 11 11:58:09 2019

@author: 123
"""

import requests
import re
import csv

url = "https://bj.lianjia.com/ershoufang/pg"
begin_page = int(input("请输入爬取起始页数:"))
end_page = int(input("请输入爬取终止页数:"))
L = list(range(begin_page,end_page+1))
headers = {
            "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3",
            "Cache-Control": "max-age=0",
            "Connection": "keep-alive",
            "Host": "bj.lianjia.com",
            "Upgrade-Insecure-Requests": "1",
            "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
        }

p = re.compile('<div class="houseInfo">.*?"region">(.*?)</a>.*?class="totalPrice"><span>(.*?)</span>万.*?单价(.*?)元/平米',re.S)
with open("链家信息.csv","a",newline="") as f:
    writer = csv.writer(f)
    writer.writerow(["小区名称","总价(万)","单价"])
for page in L:
    req = requests.get(url+str(page),headers = headers)
    req.encoding = "utf-8"
    html = req.text
    r_list = p.findall(html)
    for r_tuple in r_list:
        with open("链家信息.csv","a",newline="") as f:
            #创建写入对象
            writer = csv.writer(f)
           # L = list(r_tuple)
            L1 = [r_tuple[0].strip(),r_tuple[1].strip(),r_tuple[2].strip()]
            writer.writerow(L1)
print("爬取结束!")

这次的爬虫只是做到了能够爬到数据,代码的结构不够美观,没有封装,注释也不多,但条理足够清晰.适合新手阅读…哈哈,我也是新手.