网页抓取三种方式

正则表达式

范例

Import re
Import requests
url＝'http://example.webscraping.com/view/United-Kingdom239'
html=requests.get(url).content
re.findall（'＜td class＝'w2pfw'＞(.*?)</td＞'，html)[1]

Beautifulsoup

范例

Import requests
from bs4 import BeautifulSoup
url = 'http://example.webscraping.com/places/view/United-Kingdom-239'
html=requests.get(url).content
soup = BeautifulSoup(html,'lxml')#解析文档
tr = soup.find(attrs = {'id':'places_area__row'})#抓取元素
td = tr.find(attrs = {'class':'w2p_fw'})
area = td.text
print(area)

lxml

范例

import lxml.html
broken_html='<ul class=country <li>Area<li>Population</ul>'
tree=lxml.html.fromstring(broken_html)#获取文档
fixed_html=lxml.html.tostring(tree,pretty_print=True )#解析文档
print fixed_html
<ul class＝”country">
<li>Area</li>
<li>Population</li>
</ul>
tree=lxml.html.fromstring(html)
td=tree.cssselect（'tr#places_area_row>td.w2p_fw'）[O]
area=td.textcontent()
print area

比较：
速度：正则表达式最快，Bs最慢，lxml居中
难易：正则最难，Bs最简单，lxml居中