- 关联选择
在做选择的时候,有时候不能做到一步就选到想要的节点元素,需要先选中某一个节点元素,然后以它为基准再选择它的子节点、父节点、兄弟节点等,这里就来介绍这些节点元素。
(1)子节点和子孙节点
html = """
<html>
<head>
<title>The Dormouse's story</title>
</head>
<body>
<p class="story">
Once upon a time there were three little sisters; and their names were
<a href="http://example.com/elsie" class="sister" id="link1">
<span>Elsie</span>
</a>
<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a>
and
<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>
and they lived at the bottom of a well.
</p>
<p class="story">...</p>
"""
from bs4 import BeautifulSoup
soup=BeautifulSoup(html,"lxml")
#选取节点元素之后,如果想要获取它的直接子节点,可以调用contents属性
print(soup.p.contents)
#返回结果是以列表形式
运行结果:
#p节点里既包含文本,又包含节点,最后会将它们以列表形式统一返回。
['\n Once upon a time there were three little sisters; and their names were\n ', <a class="sister" href="http://example.com/elsie" id="link1">
<span>Elsie</span>
</a>, '\n', <a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>, ' \n and\n ', <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>, '\n
and they lived at the bottom of a well.\n ']
列表中的每个元素都是p节点的直接子节点。比如第一个a节点里面包含一层span节点,这相当于孙子节点了,但是返回结果并没有单独把span节点选出来。所以说,contents属性得到的结果是直接子节点的列表。
同样,我们可以调用children属性得到相应的结果:
print('children:',soup.p.children)
#调用了children属性来选择,返回结果是生成器类型。接下来,我们用for循环输出相应的内容。
for i,child in enumerate(soup.p.children):
print(i,child)
运行结果:
children: <list_iterator object at 0x000002810240E088>
0
Once upon a time there were three little sisters; and their names were
1 <a class="sister" href="http://example.com/elsie" id="link1">
<span>Elsie</span>
</a>
2
3 <a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>
4
and
5 <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>
6
and they lived at the bottom of a well.
from bs4 import BeautifulSoup
soup=BeautifulSoup(html,"lxml")
print('descendants:',soup.p.descendants)
#如果要得到所有的子孙节点的话,可以调用descendants属性:
for i,child in enumerate(soup.p.descendants):
print(i,child)
运行结果(此时返回结果还是生成器。输出结果包含了span节点。descendants会递归查询所有子节点,得到所有的子孙节点):
descendants: <generator object Tag.descendants at 0x0000013A32E399C8>
0
Once upon a time there were three little sisters; and their names were
1 <a class="sister" href="http://example.com/elsie" id="link1">
<span>Elsie</span>
</a>
2
3 <span>Elsie</span>
4 Elsie
5
6
7 <a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>
8 Lacie
9
and
10 <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>
11 Tillie
12
and they lived at the bottom of a well.
- 父节点和祖先节点
如果要获取某个节点元素的父节点,可以调用parent属性:
print(soup.a.parent)
这里输出的仅仅是a节点的直接父节点,而没有再向外寻找父节点的祖先节点。
如果想获取所有的祖先节点,可以调用parents属性:
print(type(soup.a.parents))
print(list(enumerate(soup.a.parents)))
返回结果是生成器对象。用列表输出了它的索引和内容,而列表中的元素就是a节点的祖先节点。
- 兄弟节点
print('Next Sibling',soup.a.next_sibling)
print('Prev Sibling',soup.a.previous_sibling)
print('Next Siblings',list(enumerate(soup.a.next_siblings)))
print('Prev Siblings',list(enumerate(soup.a.previous_siblings)))
#其中next_sibling和previous_sibling分别获取节点的下一个和上一个兄弟元素,next_siblings和previous_siblings则分别返回所有前面和后面的兄弟节点的生成器。
- 提取信息
#如果返回结果是单个节点,那么可以直接调用string、attrs等属性获得其文本和属性
print('Next Sibling:')
print(type(soup.a.next_sibling))
print(soup.a.next_sibling)
print(soup.a.next_sibling.string)
#;如果返回结果是多个节点的生成器,则可以转为列表后取出某个元素,然后再调用string、attrs等属性获取其对应节点的文本和属性。
print('Parent:')
print(type(soup.a.parents))
print(list(soup.a.parents)[0])
print(list(soup.a.parents)[0].attrs['class'])