安装beautuful soup库
在https://python123.io/ws/demo.html进行测试
bs4库的五种基本元素:
Tag 标签,
Name标签的名字,
Attributes标签的属性
NavigableString标签之间的字符串
Comment标签之间注释的字符串
bs4库的遍历功能:
遍历儿子节点和子孙节点的相关信息:
.contents
.children
.descendants
.parent 遍历父亲节点
.parents 遍历先辈节点
获取平行节点,后两个用在迭代器中
(平行节点需要在一个父亲节点下)
.next_sibling
.previous_sibling
.next_siblings
.previous_siblings
#beautuful soup的试用
import requests
from bs4 import BeautifulSoup
def getHTMLText(url):
try:
r = requests.get(url)
r.raise_for_status()
r.encoding = r.apparent_encoding
demo = r.text
return BeautifulSoup(demo, "html.parser")//以html格式进行解析
except:
return "产生异常"
if __name__ == "__main__":
url = "http://python123.io/ws/demo.html"
print(getHTMLText(url))
#prettify更好的显示标签
import requests
from bs4 import BeautifulSoup
url = "http://python123.io/ws/demo.html"
r = requests.get(url)
demo = r.text
soup = BeautifulSoup(demo, "html.parser") # 以html格式进行解析
print(soup.prettify())