丑憨批的爬虫笔记4BeautifulSoup4

pip install beautifulsoup4
https://python123.io/ws/demo.html

使用方法
参数：1.html信息2.解析器

import requests
r = requests.get('https://python123.io/ws/demo.html')
demo=r.text
from bs4 import BeautifulSoup
soup = BeautifulSoup(demo,'html.parser')
print(soup.prettify())

import requests
r = requests.get('https://python123.io/ws/demo.html')
demo=r.text
from bs4 import BeautifulSoup
soup = BeautifulSoup(demo,'html.parser')
#此时soup代表demo页面
soup.title
tag =soup.a#只能获取第一个a标签内容
#name
soup.a.name#查看a的名字
soup.a.parent.name
soup.a.parent.parent.name
#属性 是个字典
tag.attrs
tag.attrs['class']
tag.attrs['href']
type(tag.attrs)
type(tag)
#标签内容 可以跨越多个标签层次
soup.a.string
soup.p
soup.p.string
type(soup.p.string)

无法处理注释

summary

基于bs4库的HTML遍历方法

.contents用法

import requests
r = requests.get('https://python123.io/ws/demo.html')
demo=r.text
from bs4 import BeautifulSoup
soup = BeautifulSoup(demo,'html.parser')
#此时soup代表demo页面
soup.head
soup.head.contents#获得儿子节点
soup.body.contents
len(soup.body.contents)
soup.body.contents[1]#取第一个

遍历儿子节点

上行遍历

import requests
r = requests.get('https://python123.io/ws/demo.html')
demo=r.text
from bs4 import BeautifulSoup
soup = BeautifulSoup(demo,'html.parser')
#此时soup代表demo页面
soup.title.parent
soup.html.parent#是自己
soup.parent#是空的

平行遍历

标签树平行遍历有条件

import requests
r = requests.get('https://python123.io/ws/demo.html')
demo=r.text
from bs4 import BeautifulSoup
soup = BeautifulSoup(demo,'html.parser')
#此时soup代表demo页面
soup.a.next_sibling
# 标签间的string也构成平行关系，所以是字符串
soup.a.next_sibling.next_sibling

###之前的
soup.a.previous_sibling
soup.a.previous_sibling.previous_sibling#没了

summary

格式化和编码

.prettify

import requests
r = requests.get('https://python123.io/ws/demo.html')
demo=r.text
from bs4 import BeautifulSoup
soup = BeautifulSoup(demo,'html.parser')
#此时soup代表demo页面
print(soup.prettify())
print(soup.a.prettify())

编码
无
**

summary

**

标签tag
标签名Name
标签属性Attributes
标签间字符串NavigableDtring
注释的字符串Comment