【spider04】BeautifulSoup

最新推荐文章于 2024-06-29 10:14:34 发布

cs_xuhuan

最新推荐文章于 2024-06-29 10:14:34 发布

阅读量112

点赞数

分类专栏：爬虫文章标签： Python 爬虫

本文链接：https://blog.csdn.net/cs_xuhuan/article/details/82843168

版权

本文介绍了BeautifulSoup这个强大的网页解析库，详细讲解了不同解析器的优缺点，如Python标准库、lxml和html5lib。此外，还阐述了基本使用方法，包括标签选择器的各种操作，如选择元素、获取属性和内容，以及CSS选择器的应用。文章总结时推荐使用lxml解析器，并给出了使用建议。

摘要由CSDN通过智能技术生成

BeautifulSoup

灵活又方便的网页解析库，处理高效，支持多种解析器。

利用它不用编写正则表达式即可方便实现网页信息的提取。

解析器：Python标准库

使用方法：BeautifulSoup(markup, “html.parser”)

优势：Python的内置标准库执行速度适中文档容错能力强

劣势：Python 2.7.3 or 3.2.2)前的版本中文档容错能力差

解析器：lxml HTML 解析器

使用方法：BeautifulSoup(markup, “lxml”)

优势：速度快文档容错能力强

劣势：需要安装C语言库

解析器：lxml XML 解析器

使用方法：BeautifulSoup(markup, [“lxml”, “xml”]) BeautifulSoup(markup, “xml”)

优势：速度快唯一支持XML的解析器

劣势：需要安装C语言库

解析器：html5lib

使用方法：BeautifulSoup(markup, “html5lib”)

优势：最好的容错性以浏览器的方式解析文档生成HTML5格式的文档

劣势：速度慢不依赖外部拓展

基本使用

html = '''
<html><head><title>The Domouse's story</title></head>
<body>
<p class="title"name="dromouse"><b>The Dormouse's story</b></p>
<p class="story">Once upon a time there were little sisters;and their names were
<a href="http://example.com/elsie"class="sister"id="link1"><!--Elsie--></a>
<a hred="http://example.com/lacle"class="sister"id="link2">Lacle</a>and
<a hred="http://example.com/tilie"class="sister"id="link3">Tillie</a>
and they lived at bottom of a well.</p>
<p class="story">...</p>
'''

from bs4 import BeautifulSoup
soup = BeautifulSoup(html,'lxml')
print(soup.prettify())
print(soup.title.string)

<html>
 <head>
  <title>
   The Domouse's story
  </title>
 </head>
 <body>
  <p class="title" name="dromouse">
   <b>
    The Dormouse's story
   </b>
  </p>
  <p class="story">
   Once upon a time there were little sisters;and their names were
   <a class="sister" href="http://example.com/elsie" id="link1">
    <!--Elsie-->
   </a>
   <a class="sister" hred="http://example.com/lacle" id="link2">
    Lacle
   </a>
   and
   <a class="sister" hred="http://example.com/tilie" id="link3">
    Tillie
   </a>
   and they lived at bottom of a well.
  </p>
  <p class="story">
   ...
  </p>
 </body>
</html>
The Domouse's story

标签选择器

选择元素

html = '''
<html><head><title>The Domouse's story</title></head>
<body>
<p class="title"name="dromouse"><b>The Dormouse's story</b></p>
<p class="story">Once upon a time there were little sisters;and their names were
<a href="http://example.com/elsie"class="sister"id="link1"><!--Elsie--></a>
<a hred="http://example.com/lacle"class="sister"id="link2">Lacle</a>and
<a hred="http://example.com/tilie"class="sister"id="link3">Tillie</a>
and they lived at bottom of a well.</p>
<p class="story">...</p>
'''
from bs4 import BeautifulSoup
soup = BeautifulSoup(html,'lxml')
print(soup.title)
print(type(soup.title))
print(soup.head)
print(soup.p)
# 标签选择器返回选择的第一个内容

<title>The Domouse's story</title>
<class 'bs4.element.Tag'>
<head><title>The Domouse's story</title></head>
<p class="title" name="dromouse"><b>The Dormouse's story</b></p>

获取名称

html = '''
<html><head><title>The Domouse's story</title></head>
<body>
<p class="title"name="dromouse"><b>The Dormouse's story</b></p>
<p class="story">Once upon a time there were little sisters;and their names were
<a href="http://example.com/elsie"class="sister"id="link1"><!--Elsie--></a>
<a hred="http://example.com/lacle"class="sister"id="link2">Lacle</a>and
<a hred="http://example.com/tilie"class="sister"id="link3">Tillie</a>
and they lived at bottom of a well.</p>
<p class="story">...</p>
'''
from bs4 import BeautifulSoup
soup = BeautifulSoup(html,'lxml')
print(soup.title.name)#最外层标签的名称

title

获取属性

html = '''
<html><head><title>The Domouse's story</title></head>
<body>
<p class="title"name="dromouse"><b>The Dormouse's story</b></p>
<p class="story">Once upon a time there were little sisters;and their names were
<a href="http://example.com/elsie"class="sister"id="link1"><!--Elsie--></a>
<a hred="http://example.com/lacle"class="sister"id="link2">Lacle</a>and
<a hred="http://example.com/tilie"class="sister"id="link3">Tillie</a>
and they lived at bottom of a well.</p>
<p class="story">...</p>
'''
from bs4 import BeautifulSoup
soup = BeautifulSoup(html,'lxml')
print(soup.p.attrs['name'])
print(soup.p['name'])

dromouse
dromouse

获取内容

html = '''
<html><head><title>The Domouse's story</title></head>
<body>
<p class="title"name="dromouse"><b>The Dormouse's story</b></p>
<p class="story">Once upon a time there were little sisters;and their names were
<a href="http://example.com/elsie"class="sister"id="link1"><!--Elsie--></a>
<a hred="http://example.com/lacle"class="sister"id="link2">Lacle</a>and
<a hred="http://example.com/tilie"class="sister"id="link3">Tillie</a>
and they lived at bottom of a well.</p>
<p class="story">...</p>
'''
from bs4 import BeautifulSoup
soup = BeautifulSoup(html,'lxml')

最低0.47元/天解锁文章

cs_xuhuan

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
【spider04】BeautifulSoup

BeautifulSoup灵活又方便的网页解析库，处理高效，支持多种解析器。利用它不用编写正则表达式即可方便实现网页信息的提取。解析器：Python标准库使用方法：BeautifulSoup(markup, “html.parser”)优势：Python的内置标准库执行速度适中文档容错能力强劣势：Python 2.7.3 or 3.2.2)前的版本中文档容错能力差解析器：lxm...
复制链接

扫一扫

专栏目录