今天写了一个 百度图片爬虫,代码如下
import requests
import re
import os
import time
headers = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36'}
name = input('您要爬取什么图片')
num = 0
x = input('您要爬取几张呢?,输入1等于60张图片。')
for i in range(int(x)):
name_1 = 'D:\\图片\\'
url = 'https://image.baidu.com/search/flip?tn=baiduimage&ie=utf-8&word='+name+'&pn='+str(i*30)
res = requests.get(url,headers=headers)
htlm_1 = res.content.decode()
a = re.findall('"objURL":"(.*?)",',htlm_1)
if not os.path.exists(name_1):
os.makedirs(name_1)
for b in a:
num = num +1
try:
img = requests.get(b)
except Exception as e:
print('第'+str(num)+'张图片无法下载------------')
print(str(e))
continue
f = open(name_1+name+str(num)+'.jpg','ab')
print('---------正在下载第'+str(num)+'张图片----------')
f.write(img.content)
f.close()
print('下载完成')
三十行的代码就搞定了,有在学习爬虫的同学可以 参考一下,
后面会发一个教程:如何把代码封装成一个.exe可运行的文件,这样就可以自己写好代码 发给其他人 直接运行即可 不用安装插件。
------------------------------------------------
5-10
优化一下爬虫 感谢@编程副本 提出的一个小bug:会爬取重复图片。
更新如下:
import requests
import re
import os
import time
headers = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36'}
name = input('您要爬取什么图片')
num = 0
num_1 = 0
num_2 = 0
x = input('您要爬取几张呢?,输入1等于60张图片。')
list_1 = []
for i in range(int(x)):
name_1 = os.getcwd()
name_2 = os.path.join(name_1,'图片')
url = 'https://image.baidu.com/search/flip?tn=baiduimage&ie=utf-8&word='+name+'&pn='+str(i*30)
res = requests.get(url,headers=headers)
htlm_1 = res.content.decode()
a = re.findall('"objURL":"(.*?)",',htlm_1)
if not os.path.exists(name_2):
os.makedirs(name_2)
for b in a:
try:
b_1 = re.findall('https:(.*?)&',b)
b_2 = ''.join(b_1)
if b_2 not in list_1:
num = num +1
img = requests.get(b)
f = open(os.path.join(name_1,'图片',name+str(num)+'.jpg'),'ab')
print('---------正在下载第'+str(num)+'张图片----------')
f.write(img.content)
f.close()
list_1.append(b_2)
elif b_2 in list_1:
num_1 = num_1 + 1
continue
except Exception as e:
print('---------第'+str(num)+'张图片无法下载----------')
num_2 = num_2 +1
continue
print('下载完成,总共下载{}张,成功下载:{}张,重复下载:{}张,下载失败:{}张'.format(num+num_1+num_2,num,num_1,num_2))
如代码有问题欢迎大家私信或者在此贴下回复,感谢各位!
非学无以广才,非志无以成学!