python爬虫_xpath解析_ZHOU125disorder_

xpath解析

xpath解析的原理

	1. 实列化一个etree对象，并且将需要解析页面的源码加载到该对象中;
	2. 调用etree对象中的xpath方法结合这xpath表达式实现标签的定位和内容的捕获;

xpath解析环境安装

pip install lxml

• 导入etree对象

from lxml import etree

• 实例化etree对象
  1. 将本地的html文档中的源码数据加载到etree对象中

  	etree.parse(filepath)		//filepath为本地html的路径
  

from lxml import etree
if __name__ == '__main__':
    tree = etree.parse('text.html')
    r = tree.xpath('/html/body')
    print(r)

2.  `将互联网上获取的源码数据加载到etree对象中`	
```
etree.HTML(page_text)		//page_text为在互联网上获取到的相应数据
```

    page_text = requests.get(url=url, headers=headers).text
    # 数据解析
    tree = etree.HTML(page_text)
    # 存储的就是li标签对象
    li_list = tree.xpath('//ul[@class="house-list-wrap"]/li')

• xpath表达式

# 导入xpath和requests模块
from lxml import etree
if __name__ == '__main__':
    tree = etree.parse('text.html')
    # /一个层级
    r = tree.xpath('/html/div/p')
    # //多个层级
    r = tree.xpath('/html//p')
    # [@class='class_name']
    r = tree.xpath('//div[@class="tang"]')
    # [@id='id_name']
    r = tree.xpath('//div[@id="song"]')
    # 索引定位  索引是从1开始的
    r = tree.xpath('//div[@id="song"]/p[3]')
    # 获取文本  /text()获取直系文本
    r = tree.xpath('//div[@id="song"]/p[2]/text()')[0]
    # 获取文本  //text()获取非直系文本内容
    r = tree.xpath('//div[@class="tang"]//li[1]//text()')[0]
    print(r)