1、百度关键词提交
若按照MOOC上老师的代码来提交关键词,我们会遇到百度的安全验证
我们打开浏览器,搜索Python,结果如下
观察网址特点,开始编写代码
import requests
try:
# kv={'wd':'python'}
kv ={'ie':'utf-8','wd': 'Python','p_timestamp':'1583634916'}
ua = {'user-agent': 'Mozilla/5.0 (Windows NT 6.2) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1062.0 Safari/536.3'}
r=requests.get("http://www.baidu.com/s",headers=ua,params=kv)
print(r.request.url)
r.raise_for_status()
print(len(r.text))
except:
print("爬取失败")
2、360关键词提交
360关键词提交时没有安全验证,直接提交关键词即可
import requests
keyword="Python"
try:
kv={'q':keyword}
r=requests.get('http://www.so.com/s',params=kv)
print(r.request.url)
r.raise_for_status()
print(len(r.text))
except:
print("爬取失败")