基于bs4库的HTML内容遍历与输出

最新推荐文章于 2020-12-24 14:37:33 发布

星辰浩宇

最新推荐文章于 2020-12-24 14:37:33 发布

阅读量808

点赞数

分类专栏： python

本文链接：https://blog.csdn.net/amf12345/article/details/104627948

版权

python 专栏收录该内容

23 篇文章 3 订阅

订阅专栏

bs4库可以对HTML页面进行遍历，分为三种，下行遍历、上行遍历和平行遍历

标签树的下行遍历
属性	说明
.contents	子节点的列表，将<tag>所有儿子存入列表
.children	子节点的迭代类型，与.content类似，用于循环遍历儿子节点
.descendants	子孙几点的迭代类型，包含所有的子孙节点，用于循环遍历

代码如下：

import requests
from bs4 import BeautifulSoup
r = requests.get("https://python123.io/ws/demo.html")
demo = r.text
soup = BeautifulSoup(demo,"html.parser")
soup.head
<head><title>This is a python demo page</title></head>
soup.head.contents
[<title>This is a python demo page</title>]
>>> soup.body.contents
['\n', <p class="title"><b>The demo python introduces several python courses.</b></p>, '\n', <p class="course">Python is a wonderful general-purpose programming language. You can learn Python from novice to professional by tracking the following courses:

<a class="py1" href="http://www.icourse163.org/course/BIT-268001" id="link1">Basic Python</a> and <a class="py2" href="http://www.icourse163.org/course/BIT-1001870001" id="link2">Advanced Python</a>.</p>, '\n']
 len(soup.body.contents)
5
 soup.body.contents[1]
<p class="title"><b>The demo python introduces several python courses.</b></p>

通用代码如下：

for child in soup.body.children:
    print(child)  //遍历儿子节点

for child in soup.body.descendants:
    print(child)  //遍历子孙节点

上行便利

标签树的上行遍历
属性	说明
.parent	节点的父亲标签
.parents	节点先辈标签的迭代类型，用于循环遍历先辈节点

代码如下：

>>> soup.title.parent
<head><title>This is a python demo page</title></head>
>>> soup.html.parent
<html><head><title>This is a python demo page</title></head>
<body>
<p class="title"><b>The demo python introduces several python courses.</b></p>
<p class="course">Python is a wonderful general-purpose programming language. You can learn Python from novice to professional by tracking the following courses:

<a class="py1" href="http://www.icourse163.org/course/BIT-268001" id="link1">Basic Python</a> and <a class="py2" href="http://www.icourse163.org/course/BIT-1001870001" id="link2">Advanced Python</a>.</p>
</body></html>
>>> soup.parent
>>>

>>> soup.parent
>>> soup = BeautifulSoup(demo,"html.parser")
>>> for parent in soup.a.parents:
	if parent is None:
		print(parent)
	else:
		print(parent.name)

p
body
html
[document]

平行遍历：必须发生在同一父节点下的各节点间

标签树的平行遍历
属性	说明
.next.sibling	返回按照HTML文本顺序的下一个平行节点标签
.previous.sibling	返回按照HTML文本顺序的上一个平行节点标签
.next.siblings	迭代类型，返回按照HTML文本顺序的后续所有平行节点标签
.previous.siblings	迭代类型，返回按照HTML文本顺序的前续所有平行节点标签

>>> soup.a.next_sibling
' and '
>>> soup.a.next_sibling.next_sibling
<a class="py2" href="http://www.icourse163.org/course/BIT-1001870001" id="link2">Advanced Python</a>
>>> soup.a.previous_sibling
'Python is a wonderful general-purpose programming language. You can learn Python from novice to professional by tracking the following courses:\r\n'
>>> soup.a.previous_sibling.previous_sibling
>>> soup.a.parent
<p class="course">Python is a wonderful general-purpose programming language. You can learn Python from novice to professional by tracking the following courses:

<a class="py1" href="http://www.icourse163.org/course/BIT-268001" id="link1">Basic Python</a> and <a class="py2" href="http://www.icourse163.org/course/BIT-1001870001" id="link2">Advanced Python</a>.</p>
>>>

循环遍历如下：

for sibling in soup.a.next.siblings:
    print(sibling)//遍历后续节点

for sibling in soup.a.previous.siblings:
    print(sibling)//遍历前续节点

输出方法：

使用prettify函数进行美化，可以为html的文本增加换行符，也可以对某一特定的标签进行处理

bs4将任何读入的内容都转化为utf-8编码

星辰浩宇

关注

0
点赞
踩
1

收藏

觉得还不错? 一键收藏
0
评论
基于bs4库的HTML内容遍历与输出

bs4库可以对HTML页面进行遍历，分为三种，下行遍历、上行遍历和平行遍历标签树的下行遍历属性说明 .contents 子节点的列表，将<tag>所有儿子存入列表 .children 子节点的迭代类型，与.content类似，用于循环遍历儿子节点 .descendants 子孙几点的迭代类型，包含所有的子孙节点，用于循环遍历代码如下...
复制链接

扫一扫