安装requests库
我是在pycharm操作的
在终端输入以下代码
pip install requests
安装完库就可以操作了(第一个例子)
用百度作为例子
import requests
r = requests.get('https://www.baidu.com/')
print(type(r))
print(r.status_code)
print(type(r.text))
print(r.text)
print(r.cookies)
两种请求
get请求
用百度来举例子
https://www.baidu.com/
用get请求百度,输出txt文本获取百度源码
import requests
url = 'https://www.baidu.com/'
a = requests.get(url)
print(a.text)
这是结果:
反扒
先获取cookies:
import requests
headers={
'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36 QIHU 360SE'
}
url='http://www.zhihu.com'
r=requests.get(url=url,headers=headers)
print(r.cookies)
结果
或者用浏览器打开你想爬的网站右键检查开获取cookies,headers也是一样
用我们的cookies来获取网页
import requests
headers={
'Cookie':'_zap=f4cf1039-988d-4506-86b0-4a66e741c6b1; d_c0="AGDcaFGHGRKPTutiDmNxGnxfi7VhsfQ0wI8=|1603730839"; _xsrf=01xnSvUI1MkWP715R02yeXnThs2EHIXu; Hm_lvt_98beee57fd2ef70ccdd5ca52b9740c49=1610911317,1611507538,1611565882,1611566000; SESSIONID=EQPbneOhTXEKEWzoKhctFGCvXtNsbB6hgyaptDJMHfy; JOID=UFoUAUOmDkyYr9xFaaZkkCC9KVZ441wf8Mu5CQL4VgrQ4IE_BWQiVfil30VgxKKpzSBYFUbBpzXzd2z2Km1WeDs=; osd=WloUBkysDkyfoNZFaaFrmiC9Llly41wY_8G5CQX3XArQ5441BWQlWvKl30JvzqKpyi9SFUbGqD_zd2v5IG1WfzQ=; Hm_lpvt_98beee57fd2ef70ccdd5ca52b9740c49=1611673785; capsion_ticket="2|1:0|10:1611673806|14:capsion_ticket|44:N2ExMGExOTQ3YWIwNGE1YzliMTc1Mzk0ZmEwMjAyYTE=|5aecaa59c17c237af06b47a7b1402eb5b996139c8a6e1d15490899fab3c17108"; KLBRSID=031b5396d5ab406499e2ac6fe1bb1a43|1611673848|1611672766; z_c0="2|1:0|10:1611673849|4:z_c0|92:Mi4xUkFJd0lnQUFBQUFBWU54b1VZY1pFaVlBQUFCZ0FsVk4tWDc5WUFCQmZYWFB4ZkM5Z3l6ZlRNSENUUHVhR0lmYy1B|6d89241fc554ad378bce7f27715f2a4cc63cf87028c2da1e4104423b99ee14ee"; unlock_ticket="APBUrbfKXhImAAAAYAJVTQE4EGCaxoSZiXGfIktWFZReL6J3wOaKOQ=="',
'User-Agent':'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11',
'host':'www.zhihu.com',
}
url='http://www.zhihu.com'
r=requests.get(url=url,headers=headers)
print(r.text)