一、什么是BeautifulSoup
Beautiful Soup 是一个可以从HTML或XML文件中提取数据的Python库。在爬虫领域用的比较多，能够帮助我们从HTML文件中提取特定的内容，来进行分析。
二、简单实用BeautifulSoup
from bs4 import BeautifulSoup
from urllib.request import urlopen
import lxml
# 返回一个经过lxml解析的BeautifulSoup对象，
# BeautifulSoup 对象表示的是一个文档的全部内容
soup = BeautifulSoup(html, "lxml")三、BeautifulSoup中的对象
<html>
<head>
<title>
The Dormouse's story
</title>
</head>
<body>
<p class="title">
<b>
The Dormouse's story
</b>
</p>
<p class="story">
Once upon a time there were three little sisters; and their names were
<a class="sister" href="http://example.com/elsie" id="link1">
Elsie
</a>
,
<a class="sister" href="http://example.com/lacie" id="link2">
Lacie
</a>
and
<a class="sister" href="http://example.com/tillie" id="link2">
Tillie
</a>
; and they lived at the bottom of a well.
</p>
<p class="story">
...
</p>
</body>
</html>1、Tag对象
Tag 对象与XML或HTML原生文档中的tag相同。是一个Tag对象。
from bs4 import BeautifulSoup
>>> soup = BeautifulSoup('<b class="boldest">Extremely bold</b>', 'lxml')
# 返回一个tag实例，代表整个标签，包括标签的名字，属性和内容
>>> soup.b
<b class="boldest">Extremely bold</b>
>>> type(soup.b)
<class 'bs4.element.Tag'>
# tag为Tag的一个实例，通过BeautifulSoup对象获取
>>> tag = soup.b
# 获取tag对应的属性值
>>> tag["class"]
['boldest']
# 获取tag对应的内容
>>> tag.string
'Extremely bold'
# 获取tag的名字
>>> tag.name
'b'
# 获取tag的全部属性
>>> tag.attrs
{'class': ['boldest']}
2、Navigable对象
字符串就是标签的内容，这在BeautifulSoup中被称之为Navigable对象。例如：<b class="boldest">Extremely bold</b>，中对象为Extremely bold。该对象可以通过tag.string获取。
>>> soup = BeautifulSoup('<a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>', 'lxml')
>>> tag = soup.a
>>> tag.string
'Lacie'
>>> tag.name
'a'
>>> tag.attrs
{'class': ['sister'], 'href': 'http://example.com/lacie', 'id': 'link2'}
>>> tag["href"]
'http://example.com/lacie'3、BeautifulSoup对象
BeautifulSoup 对象表示的是一个文档的全部内容。通过soup.Tag可以返回一个Tag标签对象。
4、Comment对象
四、遍历文档树
子节点
一个Tag可能包含多个字符串或其它的Tag,这些都是这个Tag的子节点.Beautiful Soup提供了许多操作和遍历子节点的属性.注意: Beautiful Soup中字符串节点不支持这些属性,因为字符串没有子节点。
通过soup.tagName便可以获得该标签的全部内容。例如：
>>> html_doc = """
<html><head><title>The Dormouse's story</title></head>
<p class="title"><b>The Dormouse's story</b></p>
<p class="story">Once upon a time there were three little sisters; and their names were
<a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>,
<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
and they lived at the bottom of a well.</p>
<p class="story">...</p>
"""
# 根据tag名字寻找特定的子节点
>>> soup = BeautifulSoup(html_doc, 'lxml')
>>> soup.head
<head><title>The Dormouse's story</title></head>
>>> soup.title
<title>The Dormouse's story</title>
# 存在多个标签时，只输出第一个标签
>>> soup.a
<a class="sister" href="http://example.com/elsie" id="link1">Elsie</a>
# 通过.来取得标签内的标签
>>> soup.head.title
<title>The Dormouse's story</title>
# 取标签p时，会取到第一个标签，但是第一个标签p没有a标签，所以输出结果为None
>>> soup.p.a
>>>
>>>
# .contents属性获取子节点
>>> soup.p.contents
[<b>The Dormouse's story</b>]
>>> soup.a.contents
['Elsie']
>>> soup.html.contents
[<head><title>The Dormouse's story</title></head>, '\n', <body><p class="title"><b>The Dormouse's story</b></p>
<p class="story">Once upon a time there were three little sisters; and their names were
<a class="sister" href="http://example.com/elsie" id="link1">Elsie</a>,
<a class="sister" href="http://example.com/lacie" id="link2">Lacie</a> and
<a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>;
and they lived at the bottom of a well.</p>
<p class="story">...</p>
</body>]tag属性
# tag的 .contents 属性可以将tag的子节点以列表的方式输出:
>>> soup.p.contents
[<b>The Dormouse's story</b>]
>>> soup.a.contents
['Elsie']
#
扫一扫
2387
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
