极客学院爬虫re基本知识

最新推荐文章于 2024-07-27 17:53:21 发布

mwmoo

最新推荐文章于 2024-07-27 17:53:21 发布

阅读量414

点赞数

分类专栏：爬虫文章标签：爬虫 python3 re

本文链接：https://blog.csdn.net/mwmoo/article/details/51345605

版权

爬虫专栏收录该内容

9 篇文章 0 订阅

订阅专栏

放这里比放电脑里查看方便

极客学院爬虫re基本知识代码

# -*- coding: utf-8 -*-
"""
Created on Sat May  7 07:58:13 2016

@author: s
"""
import re

old_url = 'http://www.jikexueyuan.com/course/android/?pageNum=2'
total_page = 20

f = open(r'...\text.txt', 'rb')#'...'为path
html = f.read().decode()
#print(html)
f.close()

title = re.search('<title>(.*?)</title>', html).group(1)
#print(title)

url_links = re.findall('<a href="(.*?)">', html, re.S)
for i in url_links:
    print(i)


#ul_con = re.search('<ul>(.*?)</ul>', html, re.S).group(1)
ul_con = re.findall('<ul>(.*?)</ul>', html, re.S)[0]
#print(ul_con)
con_char = re.findall('">(.*?)</a>', ul_con, re.S)
for each_line in con_char:
    print(each_line)

for i in range(2, total_page+1):
    new_url = re.sub('pageNum=\d', 'pageNum=%d'%i, old_url, re.S)
    print(new_url)