本文基于书籍《Python网络数据采集》(Web Scraping with python)
语言:python3
系统:CentOS7
python库:Beautiful Soup4
安装BeautifulSoup4
登陆官网
https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/#id5
具体的安装方式
1.使用库BeautifulSoup4
创建BeautifulSoup对象
from urllib.request import urlopen
from bs4 import BeautifulSoup
url = urlopen("你的URL")
bsObj = BeautifulSoup(url)
bsObj 为BeautifulSoup对象
将URL指向HTML/XML文本标签转换成树形层次结构,方便提取
bsObj.标签名
直接调用标签内容
2.异常
URL异常:
无此服务器 ————urlopen返回None
有服务器但服务器无此内容————404 (一个HTTPError错误)
标签异常:
无此标签—————返回None
在无此标签情况下搜索其子标签————AttributeError
异常检测模板:
from urllib.request import urlopen
from urllib.error import HTTPError
def getTitle(url):
try:
html = urlopen(url)
except HTTPError as e:
return None
try:
bsobj = BeautifulSoup(html.read())
titel = bsobj.boby.h1
except ArrtibuteError as e:
return None
title = getTitle("")
if title == None:
print("Title could not found !")
else:
print(title)