许多人的愿望的快捷的搜图程序——爬虫

最新推荐文章于 2024-05-29 14:13:29 发布

XBL0430

最新推荐文章于 2024-05-29 14:13:29 发布

阅读量647

点赞数 1

文章标签：爬虫

本文链接：https://blog.csdn.net/XBL0430/article/details/132484467

版权

随着AI的发展，模型训练越发重要，此时数据集的作用就格外明显。面对数百张图片的需求，一个快捷的搜图程序是许多人的愿望。今天博主就给大家带来了这么一段肥肠好用的代码——爬虫。

一、爬虫是什么

爬虫是对关键词图片进行爬取搜索图片的技术，博主喜欢用它来作为数据集收集的素材来源之一。

二、爬虫的代码

这里博主不多做解释，直接上代码。大家可以直接复制粘贴。

import re
import os
import requests
import tqdm

header={'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36'}

def getImg(url,idx,path):
    img=requests.get(url,headers=header)
    file=open(path+str(idx)+'.jpg','wb')
    file.write(img.content)
    file.close()

search=input("请输入搜索内容：")
number=int(input("请输入需求数量："))
path='image/'+search+'/'
if not os.path.exists(path):
    os.makedirs(path)

bar=tqdm.tqdm(total=number)
page=0
while(True):
    if number==0:
        break
    url = 'https://image.baidu.com/search/acjson'
    params={
            "tn": "resultjson_com",
            "logid": "11555092689241190059",
            "ipn": "rj",
            "ct": "201326592",
            "is": "",
            "fp": "result",
            "queryWord": search,
            "cl": "2",
            "lm": "-1",
            "ie": "utf-8",
            "oe": "utf-8",
            "adpicid": "",
            "st": "-1",
            "z": "",
            "ic": "0",
            "hd": "",
            "latest": "",
            "copyright": "",
            "word": search,
            "s": "",
            "se": "",
            "tab": "",
            "width": "",
            "height": "",
            "face": "0",
            "istype": "2",
            "qc": "",
            "nc": "1",
            "fr": "",
            "expermode": "",
            "force": "",
            "pn": str(60*page),
            "rn": number,
            "gsm": "1e",
            "1617626956685": ""
        }
    result = requests.get(url, headers=header,params=params).json()
    url_list=[]
    for data in result['data'][:-1]:
        url_list.append(data['thumbURL'])
    for i in range(len(url_list)):
        getImg(url_list[i],60*page+i,path)
        bar.update(1)
        number-=1
        if number==0:
            break
    page+=1
print("\nfinish!")