python爬虫学习第十二天

今天学习了用Beautifulsoup函数来获取指定的节点，以及用当前结点顺藤摸瓜找到其子节点，后代节点，兄弟节点，父节点。

练习1 findAll 函数抽取只包含在 标签里的文字
还顺便把class=’red’标签里的内容也提取了

# from urllib.request import urlopen
# from bs4 import BeautifulSoup

# r = urlopen('http://www.pythonscraping.com/pages/warandpeace.html')
# bsObj = BeautifulSoup(r)
# persons = bsObj.findAll('span',{'class':'green'})
# conversasions = bsObj.findAll('span',{'class':'red'})
# for name in persons:
#   print(name.get_text())
# print('\n')
# for talks in conversasions:
#   print(talks.get_text())

练习2 查找内容匹配的html元素
查找html元素在昨天已经练习过了就是find/findall函数。
利用这两个函数的tag参数与tagAtrribute参数可以让我们检索大多数标签，此外我们还可以通过text参数（下面的例子正是如此）匹配内容包含制定字符串的标签

# from urllib.request import urlopen
# from bs4 import BeautifulSoup

# r = urlopen('http://www.pythonscraping.com/pages/warandpeace.html')
# bsObj = BeautifulSoup(r)
# test = bsObj.findAll(text = 'the prince')
# print(len(test))

练习3 子标签和后代标签 注意他们的区别

子标签就是一个父标签的下一级，而后代标签是指一个父标签 下面所有级别的标签。所有的子标签都是后代标 签，但不是所有的后代标签都是子标签。

# from urllib.request import urlopen
# from bs4 import BeautifulSoup

# r = urlopen('http://www.pythonscraping.com/pages/page3.html')

# bsObj = BeautifulSoup(r)
# for child in bsObj.find('table',{'id':'giftList'}).children:
#   print(child)
# print('\n')
# for descendant in bsObj.find('table',{'id':'giftList'}).descendants:
#   print(descendant)

练习4 用next_siblings获取兄弟节点

# from urllib.request import urlopen
# from bs4 import BeautifulSoup

# r = urlopen('http://www.pythonscraping.com/pages/page3.html')
# bsObj = BeautifulSoup(r)
# for sibling in bsObj.find('table',{'id':'giftList'}).tr.next_siblings:
#   print(sibling)

练习5 用parent/parents操作父节点

# from urllib.request import urlopen
# from bs4 import BeautifulSoup

# r = urlopen('http://www.pythonscraping.com/pages/page3.html')
# bsObj = BeautifulSoup(r)
# money = bsObj.find('img',{'src':'../img/gifts/img1.jpg'}).parent.previous_sibling
# print(money.get_text())