还在为一张张的点下载图片而烦恼吗?请用一个程序员的思路来解决这个问题,下面就是可以节省你大量时间的代码;
存储媒体文件有两种方式:一是只获取URL链接;二是直接把源文件下载下来
下面这个就是直接把源文件下载下来的实例:
其中,
urlretrieve()函数用于下载文件
代码如下:
import os from urllib.request import urlretrieve from urllib.request import urlopen from bs4 import BeautifulSoup downloadDirectory = "D:\downloaded" baseUrl = "http://pythonscraping.com" def getAbsoluteURL(baseUrl, source): if source.startswith("http://www."): url = "http://" + source[11:] elif source.startswith("http://"): url = source elif source.startswith("www."): url = source[4:] url = "http://" + source else: url = baseUrl + "/" + source if baseUrl not in url: return None return url def getDownloadPath(baseUrl, absoluteUrl, downloadDirectory): path = absoluteUrl.replace("www.", "") path = path.replace(baseUrl, "") path = downloadDirectory + path directory = os.path.dirname(path) if not os.path.exists(directory): os.makedirs(directory) return path html = urlopen("http://www.pythonscraping.com") bsObj = BeautifulSoup(html) downloadList = bsObj.findAll(src=True) for download in downloadList: fileUrl = getAbsoluteURL(baseUrl, download["src"]) if fileUrl is not None: print(fileUrl) urlretrieve(fileUrl, getDownloadPath(baseUrl, fileUrl, downloadDirectory))
代码来自《python:网络爬虫与信息采取》,感觉有用的话就收藏吧