python将某个图片url爬取到本地的几种方法

python网络爬虫--从网上下载图片的方法

结合网上学习和自己动手验证，发现以下几个方法在不同情况下可用

1.使用contextlib库中的closing

from contextlib import closing
down_url = 'https://........jpg'
local_dir = 'local_file_name.jpg'

with closing(requests.get(url=down_url, stream=True)) as urlfilehandle:
    with open(local_dir, 'ab+') as localfilehandle:
        for chunk in urlfilehandle.iter_content(chunk_size=1024):
            if chunk:
                localfilehandle.write(chunk)
                localfilehandle.flush()

 

2.直接使用urllib

urllib.urlretrieve(imgURL, localFileName)

3.在存到本地时，如果无法完整下载图片或图片在本地无法打开，需添加headers，并使用urllib2库

headers = {
    'User-Agent':'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:69.0) Gecko/20100101 Firefox/69.0',
}
req = urllib2.Request(imgURL, None, headers)
response = urllib2.urlopen(req)
with open(fname, 'wb') as f:
    f.write(response.read())