举个栗子,试着爬一下宝可梦图鉴的图片,在https://cn.portal-pokemon.com/play/pokedex/
数据是前后端分离的便于获取数据。
在PM图鉴上取到json数据,形容下图,file_name是我们需要的图片地址,但是缺少文件头
image.png
于是查看图片,发现图片的前部拼接头。
image.png
于是把图片前部拼接头加上域名前缀,就获得了图片的原始路径,形如https://cn.portal-pokemon.com/play/resources/pokedex/img/pm/cf47f9fac4ed3037ff2a8ea83204e32aff8fb5f3.png
这边采用的方法先决定用保存的json文件,提取数据保存,再做图片下载,代码如下:
import json
构造对象
class pokemons(object):
def init(self,zukan_id,zukan_sub_id,pokemon_name,pokemon_sub_name,weight,height,file_name,pokemon_type_id,pokemon_type_name):
self.zukan_id = zukan_id
self.zukan_sub_id = zukan_sub_id
self.pokemon_name = pokemon_name
self.pokemon_sub_name = pokemon_sub_name
self.weight = weight
self.height = height
self.file_name = file_name
self.pokemon_type_id = pokemon_type_id
self.pokemon_type_name = pokemon_type_name
打开json文件,解析file_name
with open('pp1.json', 'r', encoding='utf-8') as f:
data = json.load(f)
拼接图片文件头,存放在list里面
headurl = 'https://cn.portal-pokemon.com/play/resources/pokedex'
result = ""
for i in range(len(data)):
data[i]['file_name'] = headurl + data[i]['file_name']
result = result + data[i]['file_name'] + '\n'
把读取到的result写入文件中
with open('bf.txt', 'a', encoding='utf-8') as bf:
print(bf.write(result))
提取数据后,保存的文件效果
image.png