3.32 小猪短租的爬虫-

 够造主网页的url获取函数，从主网页中获取到详情页的链接，从详情页中获取到标题，价格，评论等内容

 

这里因为ip反爬，返回的是错误的网页，所以后续失败

#这里进行了反爬，返回的网页进行了重定向，不是自己要爬的网址

from bs4 import BeautifulSoup
import requests
import time			#导入相应的库文件
url ="https://bj.xiaozhu.com/fangzi/1047842478.html"	
headers = {
        "Cookie": "abtest_ABTest4SearchDate=b; sajssdk_2015_cross_new_user=1; distinctId=17663eb00672c9-0d67d3dfd2265d-e726559-2073600-17663eb006841a; Hm_lvt_92e8bc890f374994dd570aa15afc99e1=1607994115,1608023687; xzuuid=87961465; xzuinfo=%7B%22user_id%22%3A153018699197%2C%22user_name%22%3A%2217317126846%22%2C%22user_key%22%3A%223d865d010085%22%2C%22user_nickName%22%3A%22wangwangluo123%22%7D; xzucode=1e98f258b6137a484cf910d72d023371; xzucode4im=ac7725f797e9e2a2b0ad8cdbe1351291; xztoken=WyIwMTA1MTIyNjE1V0xoRCIseyJ1c2VyaWQiOjE1MzAxODY5OTE5NywiZXhwaXJlIjowLCJjIjoid2ViIn0s