1. 课本
2. 弄清楚request模块的语法
>>> import webbrowser
>>> import requests
>>> res=requests.get('https://user.qzone.qq.com/3119549750')
>>> type(res)
<class 'requests.models.Response'>
>>> len(res.text)
15846
>>> print(res.text[:250])
<!DOCTYPE HTML>
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="zh-cn" lang="zh-cn">
<head>
<meta charset="utf-8" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>QQ空间</title>
<script type="
或者用这种方式检查网页是否下载成功:
>>> res.raise_for_status()
没反应就是下载成功啦~
3. 打开chorm浏览器,打开网页,按F12查看源代码
不知道为什么我的电脑按F12只能调亮度,所以我还是右键空白处选择“查看源代码”来做的。。。
and百度上说这样做之前要先右键空白处选择“审查元素”再选“查看源代码”??
4. 学习html语法
http://www.w3school.com.cn/tags/index.asp
5. 利用request中get方法下载这个网址的源代码,并且保存到你电脑文件中,打卡截图
>>> import requests
>>> res=requests.get('http://www.52nlp.cn/')
>>> res.raise_for_status()
>>> playfile=open('C:\\Users\\DELL\\Desktop\\html.txt','wb')
>>> for chunk in res.iter_content(100000):
playfile.write(chunk)
100000
7807
>>> playfile.close()
6. 弄清楚beautifulsoup模块的语法
鼓捣了一下午中午爬了一个正常的东西~
>>> res=requests.get('https://todays-weddings.com/planning/readings/poetry/love/')
>>> res.raise_for_status()
>>> love=bs4.BeautifulSoup(res.text)
>>> type(love)
<class 'bs4.BeautifulSoup'>
>>> elems=love.select('div>p>span')
>>> type(elems)
<class 'list'>
>>> len(elems)
48
>>> for i in range(48):
elems[i].getText()
7. 实习下载漫画XKCD的任务,打开截图~
import requests, os, bs4
url = 'https://xkcd.com' # starting url
os.makedirs('xkcd', exist_ok=True) # store comics in ./xkcd
while not url.endswith('#'):
# Download the page.
print('Downloading page %s...' % url)
res = requests.get(url)
res.raise_for_status()
soup = bs4.BeautifulSoup(res.text)
# Find the URL of the comic image.
comicElem = soup.select('#comic img')
if comicElem == []:
print('Could not find comic image.')
else:
comicUrl = 'https:' + comicElem[0].get('src')
# Download the image.
print('Downloading iamge %s...' % (comicUrl))
res = requests.get(comicUrl)
res.raise_for_status()
# Save the image to ./xkcd
imageFile = open(os.path.join('xkcd', os.path.basename(comicUrl)), 'wb')
for chunk in res.iter_content(100000):
imageFile.write(chunk)
# Get the Prev button's url.
prevLink = soup.select('a[rel="prev"]')[0]
url = 'http://xkcd.com' + prevLink.get('href')
print('Done')