趣味学python(016)

python——爬取猫眼电影排行(正则)

今天小编学习了正则表达式,特地用正则表达式爬取了猫眼电影排名。特来分享一下。

用到的库 requests re(正则表达式)

导入用到的库:

import requests
import re
import os

思路:
1 访问网址,拿到html数据
2 通过正则匹配 得到所需数据
3 进行数据的保存

首先构造main()函数:

def main():

>url = "https://maoyan.com/board/4?offset=0"
html = get_page(url)
parse_page(html)

构造请求函数:

def get_page(url):
headers = {
‘User-Agent’: ‘Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/75.0.3770.142 Safari/537.36’
}
response = requests.get(url, headers=headers)
#进行异常处理
if response.status_code == 200:
return response.text
return None

def parse_page(html):
patten = re.compile(’

.?board-index.?>(.?).?name.?a.?>(.?).?class=“star”.?>(.?)


.?class=“releasetime”.?>(.?).?integer.?>(.?).?fraction.?>(.?).?’, re.S)
items = re.findall(patten, html)
for item in items:
if os.path.exists(“猫眼电影排行”)==False:
os.mkdir(“猫眼电影排行”)
with open(“猫眼电影排行” +"\" + item[1], ‘w’, encoding=“utf-8”) as f:
f.write(str(item))

下面附上完整代码:

import requests
import re
import os

def get_page(url):
headers = {
‘User-Agent’: ‘Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/75.0.3770.142 Safari/537.36’
}
response = requests.get(url, headers=headers)
if response.status_code == 200:
return response.text
return None

def parse_page(html):
patten = re.compile(’

.?board-index.?>(.?).?name.?a.?>(.?).?class=“star”.?>(.?)


.?class=“releasetime”.?>(.?).?integer.?>(.?).?fraction.?>(.?).?’, re.S)
items = re.findall(patten, html)
for item in items:
if os.path.exists(“猫眼电影排行”)==False:
os.mkdir(“猫眼电影排行”)
with open(“猫眼电影排行” +"\" + item[1], ‘w’, encoding=“utf-8”) as f:
f.write(str(item))

def main():

>url = "https://maoyan.com/board/4?offset=0"
html = get_page(url)
parse_page(html)

if name == ‘main’:
main()

发布了28 篇原创文章 · 获赞 0 · 访问量 666
展开阅读全文

没有更多推荐了,返回首页

©️2019 CSDN 皮肤主题: 大白 设计师: CSDN官方博客

分享到微信朋友圈

×

扫一扫,手机浏览