python美味汤的安装

最新推荐文章于 2023-05-14 09:00:00 发布

Amitabha@python

最新推荐文章于 2023-05-14 09:00:00 发布

阅读量773

点赞数 1

分类专栏： python 文章标签： bs4 遍历 soup BeautifulSoup

本文链接：https://blog.csdn.net/qq_14983349/article/details/102007555

版权

python 专栏收录该内容

12 篇文章 0 订阅

订阅专栏

pip install beautifulsoup4
升级pip install --upgrade beautifulsoup4
打开
https://python123.io/ws/demo.html
beautifulsoup4用于解析html页面

>>> import requests
>>> r=requests.get('https://python123.io/ws/demo.html')
>>> r.status_code
200
>>> r.text
'<html><head><title>This is a python demo page</title></head>\r\n<body>\r\n<p class="title"><b>The demo python introduces several python courses.</b></p>\r\n<p class="course">Python is a wonderful general-purpose programming language. You can learn Python from novice to professional by tracking the following courses:\r\n<a href="http://www.icourse163.org/course/BIT-268001" class="py1" id="link1">Basic Python</a> and <a href="http://www.icourse163.org/course/BIT-1001870001" class="py2" id="link2">Advanced Python</a>.</p>\r\n</body></html>'
>>> demo=r.text
>>> from bs4 import BeautifulSoup
>>> soup=BeautifulSoup(demo,'html.parser')
>>> print(soup.prettfy())
Traceback (most recent call last):
  File "<pyshell#7>", line 1, in <module>
    print(soup.prettfy())
TypeError: 'NoneType' object is not callable
>>> print(soup.prettify())
<html>
 <head>
  <title>
   This is a python demo page
  </title>
 </head>
 <body>
  <p class="title">
   <b>
    The demo python introduces several python courses.
   </b>
  </p>
  <p class="course">
   Python is a wonderful general-purpose programming language. You can learn Python from novice to professional by tracking the following courses:
   <a class="py1" href="http://www.icourse163.org/course/BIT-268001" id="link1">
    Basic Python
   </a>
   and
   <a class="py2" href="http://www.icourse163.org/course/BIT-1001870001" id="link2">
    Advanced Python
   </a>
   .
  </p>
 </body>
</html>

用法

>>> from bs4 import BeautifulSoup
>>> soup=BeautifulSoup(demo,'html.parser')

>>> import requests
>>> r=requests.get("https://www.baidu.com")
r
>>> r.status_code
200
>>> demo=r.text
>>> from bs4 import BeautifulSoup
>>> soup=BeautifulSoup(demo,'html.parser')
>>> print
<built-in function print>
>>> print(soup.prettify())

BeatifulSoup库解析器
bs4的html解析器：‘html.parser’
lxml的html解析器：‘lxml’ pip install lxml
lxml的xml解析器：‘xml’ pip install lxml
html5lib的解析器：‘html5lib’ pip install html5lib

Beautiful Soup类的基本元素

基本元素	说明
Tag	标签，最基本的信息组织单元，分别用<>和</>标明开头和结尾
Name	标签的名字，<p>…</p>的名字是‘p’，格式：<tag>.name
Attibutes	标签的属性，字典形式组织，格式：<tag>.attrs
NavigableString	标签内非属性字符串，<>…</>中字符串，格式：<tag>.string
Comment	标签内字符串的注释部分，一种特殊的Comment类型

获得tag标签的方法

>>> soup.title
<title>xxx</title>
>>> tag=soup.a
>>> print(tag)
<a href="https://www.baidu.com/">
xxx
</a>
>>> soap.a.name
Traceback (most recent call last):
  File "<pyshell#28>", line 1, in <module>
    soap.a.name
NameError: name 'soap' is not defined
>>> soup.a.name
'a'
>>> soup.a.parent.name
'div'
>>> soup.a.parent.parent.name
'div'
>>> tag.attrs
{'href': 'https://www.busjav.icu/'}
>>> tag=soup.div
>>> print(div)
Traceback (most recent call last):
  File "<pyshell#34>", line 1, in <module>
    print(div)
NameError: name 'div' is not defined
>>> print(tag)

>>> tag.attrs
{'id': 'search-loading'}
>>> tag.div.string
'搜尋中...'
>>> soup.td

>>> soup.td.string
>>> print(soup.td.strin)
None
>>> print(soup.td.string)
None
>>> soup.table.attrs
{'class': ['search-loading-box'], 'border': '0', 'cellpadding': '0', 'cellspacing': '0'}
>>>

网页遍历
下行
上行
平行

>>> soup.head

>>> soup.body

>>> sooup.body.contents
Traceback (most recent call last):
  File "<pyshell#46>", line 1, in <module>
    sooup.body.contents
NameError: name 'sooup' is not defined
>>> soup.body.contents#下行遍历

>>> soup.body.contents[1]

>>> len(soup.body.contents)
46
>>> soup.body.contents[4]
'\n'
>>> soup.a.parent

>>> soup.a.parents#上行遍历
<generator object PageElement.parents at 0x00000000041272C8>
>>> for parent in soup.a.parents:
	if parent is None:
		print(parent)
	else:
		print(parent.name)

div
div
nav
body
html
[documen

标签树的下行遍历

属性	说明
.contents	子节点的列表，将<tag>所有儿子节点存入列表
.children	子节点的迭代类型，与 .contents，用于循环遍历儿子节点
.descendants	子孙节点的迭代类型，包含所有子孙节点，用于循环遍历

标签树的上行遍历

属性	说明
.parent	节点的父亲标签
.parents	节点先辈标签的迭代类型，用于循环遍历先辈节点

标签树的平行遍历
前提是，两标签来自于同一个父节点

属性	说明
.next_sibling	返回按照HTML文本顺序的下一个平行节点标签
.previous_sibling	返回按照HTML文本顺序的上一个平行节点标签
.netx_siblings	迭代类型，返回按照HTML文本顺序的后续所有平行节点标签
.previous_siblings	迭代类型，返回按照HTML文本顺序的前续所有平行节点标签

平行遍历

>>> r=requests.get('https://python123.io/ws/demo.html')
>>> r.status_code
200
>>> soup=BeautifulSoup(r.text,'html.parser')
>>> soup
<html><head><title>This is a python demo page</title></head>
<body>
<p class="title"><b>The demo python introduces several python courses.</b></p>
<p class="course">Python is a wonderful general-purpose programming language. You can learn Python from novice to professional by tracking the following courses:

<a class="py1" href="http://www.icourse163.org/course/BIT-268001" id="link1">Basic Python</a> and <a class="py2" href="http://www.icourse163.org/course/BIT-1001870001" id="link2">Advanced Python</a>.</p>
</body></html>
>>> soup.a
<a class="py1" href="http://www.icourse163.org/course/BIT-268001" id="link1">Basic Python</a>
>>> soup.a.next_sibling
' and '
>>> soup.a.next_sibling.next_sibling
<a class="py2" href="http://www.icourse163.org/course/BIT-1001870001" id="link2">Advanced Python</a>
>>> soup.a.previous_sibling
'Python is a wonderful general-purpose programming language. You can learn Python from novice to professional by tracking the following courses:\r\n'
>>> soup.a.previous_sibling.previous_sibing
Traceback (most recent call last):
  File "<pyshell#21>", line 1, in <module>
    soup.a.previous_sibling.previous_sibing
  File "D:\Python37_64\lib\site-packages\bs4\element.py", line 646, in __getattr__
    self.__class__.__name__, attr))
AttributeError: 'NavigableString' object has no attribute 'previous_sibing'
>>> for sibling in soup.a.next_siblings:
	print(sibling)

	
 and 
<a class="py2" href="http://www.icourse163.org/course/BIT-1001870001" id="link2">Advanced Python</a>
.
>>> for sibling in soup.a.previous_sibling:
	print(sibling)

Amitabha@python

关注

1
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
python美味汤的安装

pip install beautifulsoup4升级pip install --upgrade beautifulsoup4打开https://python123.io/ws/demo.htmlbeautifulsoup4用于解析html页面>>> import requests>>> r=requests.get('https://python...
复制链接

扫一扫

专栏目录