python爬虫学习教程,短短25行代码批量下载豆瓣妹子图片、非常简短,代码不是很多非常适合新手练习!
学习python、python爬虫过程中有不懂的可以加入我的python零基础系统学习交流秋秋qun:前面是934,中间109,后面是170,与你分享Python企业当下人才需求及怎么从零基础学习Python,和学习什么内容。相关学习视频资料、开发工具都有分享!
代码展示:
#!/usr/bin/env python
import urllib.request
from bs4 import BeautifulSoup
def crawl(url):
headers = {'User-Agent':'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6'}
req = urllib.request.Request(url, headers=headers)
page = urllib.request.urlopen(req, timeout=20)
contents = page.read()
soup = BeautifulSoup(contents)
my_girl = soup.find_all('img')
for girl in my_girl:
link = girl.get('src')
print(link)
content2 = urllib.request.urlopen(link).read()
w