批量下载GitHub中项目的图片 | 爬虫

举例详解。

#!/usr/bin/env python
# -*- coding:utf-8 -*-
# Author:LingInHeart

import requests,re,os
r=requests.get('https://github.com/MiracleYoung/You-are-Pythonista/tree/master/PythonExercise/App/plan_game/material_images')
urls=re.findall(r'MiracleYoung/You-are-Pythonista/blob/master/PythonExercise/App/plan_game/material_images/\w+\.png',r.text)
for v,url in enumerate(urls):
    url='https://raw.githubusercontent.com/'+url.replace('/blob','')
    path = 'D://pict/' + url.split('/')[-1]
    r = requests.get(url)
    with open(path, 'wb')as f:
        f.write(r.content)
        f.close()
        print('第%d张图片保存成功！' % (v + 1))

第一步，浏览器找到图片库的链接

点开一个图片观察链接：https://github.com/MiracleYoung/You-are-Pythonista/blob/master/PythonExercise/App/plan_game/material_images/again.png

r=requests.get('https://github.com/MiracleYoung/You-are-Pythonista/tree/master/PythonExercise/App/plan_game/material_images')
urls=re.findall(r'MiracleYoung/You-are-Pythonista/blob/master/PythonExercise/App/plan_game/material_images/\w+\.png',r.text)

获得网页源代码，通过re表达式提取所有的图片链接。for i in urls:print(i)

第二步，找到图片的源代码，观察链接：
https://raw.githubusercontent.com/MiracleYoung/You-are-Pythonista/master/PythonExercise/App/plan_game/material_images/again.png

易知urls中的所有链接都缺少前置https://raw.githubusercontent.com/ 以及多了 /blob，

url='https://raw.githubusercontent.com/'+url.replace('/blob','')

对url进行处理，添加前缀并删去多余部分。

第三步，保存图片

    path = 'D://pict/' + url.split('/')[-1]
    r = requests.get(url)
    with open(path, 'wb')as f:
        f.write(r.content)
        f.close()

下图是完整代码执行结果。

以上。