Python-爬虫-bs4入门

安装beautuful soup库
在https://python123.io/ws/demo.html进行测试

bs4库的五种基本元素:
Tag 标签,
Name标签的名字,
Attributes标签的属性
NavigableString标签之间的字符串
Comment标签之间注释的字符串

bs4库的遍历功能:
遍历儿子节点和子孙节点的相关信息:
.contents
.children
.descendants

.parent 遍历父亲节点
.parents 遍历先辈节点

获取平行节点,后两个用在迭代器中
(平行节点需要在一个父亲节点下)
.next_sibling
.previous_sibling
.next_siblings
.previous_siblings

#beautuful soup的试用
import requests
from bs4 import BeautifulSoup
def getHTMLText(url):
    try:
        r = requests.get(url)
        r.raise_for_status()
        r.encoding = r.apparent_encoding
        demo = r.text
        return BeautifulSoup(demo, "html.parser")//以html格式进行解析
    except:
        return "产生异常"


if __name__ == "__main__":
        url = "http://python123.io/ws/demo.html"
        print(getHTMLText(url))

#prettify更好的显示标签
import requests
from bs4 import BeautifulSoup

url = "http://python123.io/ws/demo.html"
r = requests.get(url)
demo = r.text
soup = BeautifulSoup(demo, "html.parser") # 以html格式进行解析
print(soup.prettify())