参考
https://www.w3school.com.cn/xpath/xpath_syntax.asp
https://www.cnblogs.com/pythonywy/p/11082153.html
常用规则
- text() 用于获取标签文本内容
- @ 属性名 用于获取对应属性的值
实例
以这个网页为例:https://www.biqubao.com/book/17587/
- 导入模块
import requests
from lxml import etree
html = requests.get('https://www.biqubao.com/book/17587/',headers=headers)
html.encoding = 'gbk'
doc = etree.HTML(html.text)
- 获取第一章的href,标题
content1 = doc.xpath( '//*[@id="list"]/dl/dd[1]/a/@href' )
content2 = doc.xpath( '//*[@id="list"]/dl/dd[1]/a/text()' )
- 获取所有章节href
content_href = doc.xpath( '//*[@id="list"]/dl/dd/a/@href' )
- 获取标题的名称
content_ul = doc.xpath( '//ul/li/a/text()' )
- 获取前面5章
content_text = doc.xpath( '//*[@id="list"]/dl/dd[position()<=5]/a/text()' )