黑板客爬虫闯关-第一关

2017/01/17

from urllib.request import urlopen
from bs4 import BeautifulSoup
import re

url = "http://www.heibanke.com/lesson/crawler_ex00/"
html = urlopen(url)
bs0bj = BeautifulSoup(html)

# <h3>你需要在网址后输入数字30867</h3>
h3 = bs0bj.h3
h3s = str(h3)
# type(h3s)
# print(h3s)

num = re.findall(r'[0-9]{5}', h3s)
# print(num)

while len(num) > 0 :
    url = "http://www.heibanke.com/lesson/crawler_ex00/"
    url = url + num[0]
    print(url)
    html = urlopen(url)
    bs0bj = BeautifulSoup(html)

    h3 = bs0bj.h3
    h3s = str(h3)

    num = re.findall(r'[0-9]{5}', h3s)

print(html)

关键点：
1. bs0bj.h3的type不是str，必须要强制转换
2. re.findall(正则表达式，检索str）
3. h3本身也含有数字3，所以要限制数字长度
4. num是list，所以必须读取其中的元素，才能接在url后面
5. if 与 while的不同