学习笔记：Python爬虫之BeautifulSoup库

最新推荐文章于 2024-04-27 17:30:34 发布

VIP文章 pythonputao

最新推荐文章于 2024-04-27 17:30:34 发布

阅读量400

点赞数

分类专栏： python 文章标签：列表 python xml java html

本文链接：https://blog.csdn.net/pythonputao/article/details/111277479

版权

1. BeautifulSoup

1.1 解析库

1）Python标准库

# 使用方法
BeautifulSoup(markup, "html.parser")

# 优势
Python的内置标准库，执行速度适中，文档容错能力强

# 劣势
Python2.7.3 或者 python3.2.2 前的版本容错能力差

2）lxml HTML解析器

绝大部分场景都应该使用lxml解析器

# 使用方法
BeautifulSoup(markup, "lxml")

# 优势
速度快，文档容错能力强

# 劣势
需要安装C语言库

3）lxml XML解析器

# 使用方法
BeautifulSoup(markup, "xml")

# 优势
速度快，唯一支持XML的解析器

# 劣势
需要安装C语言库

4）html5lib

# 使用方法
BeautifulSoup(markup, "html5lib")

# 优势
最好的容错性，以浏览器的方式解析文档，生成HTML5格式的文档

# 劣势
速度慢，不依赖外部扩展

1.2 基本使用

html = """
<html><head><title>The Dormouse's story</title></head>
<body>
<p class="title" name="dromouse"><b>The Dormouse's story</b></p>
<p class="story">Once upon a time there were three little sisters; and their names were
<a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>,
<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
and they lived at the bottom of a well.</p>
<p class="story">...</p>
"""
from bs4 import BeautifulSoup
soup = BeautifulSoup(html, 'lxml') # 使用lxml解析器
print(soup.prettify())    # 格式化代码，能自动将缺失的代码进行补全并进行容错处理
print(soup.title.string)  # 拿到title标签，并拿到其中的内容

2. 标签选择器

2.1 选择元素

可以直接通过 .标签名 的方式来选择标签

html = """
<html><head><title>The Dormouse's story</title></head>
<body>
<p class="title" name="dromouse"><b>The Dormouse's story</b></p>
<p class="story">Once upon a time there were three little sisters; and their names were
<a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>,
<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
and they lived at the bottom of a well.</p>
<p class="story">...</p>
"""
from bs4 import BeautifulSoup
soup = BeautifulSoup(html, 'lxml')
print(soup.title)        # 选择title标签，打印结果：<title>The Dormouse's story</title>
print(type(soup.title))  # 类型：<class 'bs4.element.Tag'>
print(soup.head) 
print(soup.p) # 如果有多个匹配结果，那么它只会返回第一个

2.2 获取名称

获取标签的名称，如是p标签还是a标签等

html = """
<html><head><title>The Dormouse's story</title></head>
<body>
<p class="title" name="dromouse"><b>The Dormouse's story</b></p>
<p class="story">Once upon a time there were three little sisters; and their names were
<a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>,
<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
and they lived at the bottom of a well.</p>
<p class="story">...</p>
"""
from bs4 import BeautifulSoup
soup = BeautifulSoup(html, 'lxml')
print(soup.title.name) # 获取标签名称

2.3 获取属性

可以通过 attrs["name"] 或者 标签["name"] 的方式来获取标签中name属性的值

html = """
<html><head><title>The Dormouse's story</title></head>
<body>
<p class="title" name="dromouse"><b>The Dormouse's story</b></p>
<p class="story">Once upon a time there were three little sisters; and their names were
<a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>,
<a href="http:/

最低0.47元/天解锁文章

pythonputao

关注

0
点赞
踩
2

收藏

觉得还不错? 一键收藏
0
评论
学习笔记：Python爬虫之BeautifulSoup库

1. BeautifulSoup1.1 解析库1）Python标准库# 使用方法BeautifulSoup(markup, "html.parser")# 优势Python的内置标准库，执行速度适中，文档容错能力强# 劣势Python2.7.3 或者 python3.2.2 前的版本容错能力差2）lxml HTML解析器绝大部分场景都应该使用lxml解析器# 使用方法BeautifulSoup(markup, "lxml")# 优势速度快，文档容错能力强
复制链接

扫一扫