异常处理代码框架
import requests
def getHtmlText(url):
try:
r=requests.get(url,timeout=30)
r.raise_for_status()/不是200 引发httperror
r.encoding=r.apparent_encoding
return r.text
except:
return "产生异常"
if __name__ =="__main__":
url="http://www.ba23idu1.com"
print(getHtmlText(url))
r在这是response类型的对象
Http 协议:
**HTTP**,Hypertext Transfer Protocol,超文本传输协议
HTTP是一个基于“请求与响应”模式的、无状态的应用层协议
HTTP协议采用URL作为定位网络资源的标识,URL格式如下:
http://host[:port][path]
host: 合法的Internet主机域名或IP地址
port: 端口号,缺省端口为80
path: 请求资源的路径
requests库的七个主要方法:
用法:
head:
import requests
r=requests.head("http://www.baidu.com")
print(r.headers)
print(r.text)
/*
{'Cache-Control': 'private, no-cache, no-store, proxy-revalidate, no-transform', 'Connection': 'keep-alive', 'Content-Encoding': 'gzip', 'Content-Type': 'text/html', 'Date': 'Fri, 07 May 2021 01:25:26 GMT', 'Last-Modified': 'Mon, 13 Jun 2016 02:50:08 GMT', 'Pragma': 'no-cache', 'Server': 'bfe/1.0.8.18'}
*/
post
payload ={'key1':'value1','key2':'value2'}
r=requests.post(url,data=payload)
或者
r=requests.post(url,data='abc')
// 向url post一个字符串 自动编码为data
put
payload = {'key1': 'value1', 'key2': 'value2'}
r = requests.put(url, data = payload)
requests.request(method, url, **kwargs)
-
method 请求方式,对应get/put/post等7种
-
url 拟获取页面的url链接
-
**kwargs 控制访问的参数,共13个
method: get head post put patch delete options
**kwargs:
params : 字典或字节序列,作为参数增加到url中
kv = {'key1': 'value1', 'key2': 'value2'}
r = requests.request('GET', 'http://python123.io/ws', params=kv)
data : 字典、字节序列或文件对象,作为Request的内容
kv = {'key1': 'value1', 'key2': 'value2'}
r = requests.request('POST', 'http://python123.io/ws', data=kv)
json : JSON格式的数据,作为Request的内容
kv = {'key1': 'value1'}
r = requests.request('POST', 'http://python123.io/ws', json=kv)
headers : 字典,HTTP定制头
hd = {'user‐agent': 'Chrome/10'}
r = requests.request('POST', 'http://python123.io/ws', headers=hd)
cookies : 字典或CookieJar,Request中的cookie
auth : 元组,支持HTTP认证功能
files : 字典类型,传输文件
fs = {'file': open('data.xls', 'rb')}
r = requests.request('POST', 'http://python123.io/ws', files=fs)
timeout : 设定超时时间,秒为单位
r= requests.request('GET', 'http://www.baidu.com', timeout=10)
proxies : 字典类型,设定访问代理服务器,可以增加登录认证
pxs = { 'http': 'http://user:pass@10.10.10.1:1234'
'https': 'https://10.10.10.1:4321' }
r = requests.request('GET', 'http://www.baidu.com', proxies=pxs)
allow_redirects : True/False,默认为True,重定向开关
stream : True/False,默认为True,获取内容立即下载开关
verify : True/False,默认为True,认证SSL证书开关
cert : 本地SSL证书路径
requests.get(url, params=None, **kwargs)
requests.head(url, **kwargs)
requests.post(url, data=None, json=None, **kwargs)
requests.put(url, data=None, **kwargs)
requests.patch(url, data=None, **kwargs)
requests.delete(url, **kwargs)
requests.get(url, params=None, **kwargs)
模拟浏览器
import** requests
kv={'user-agent':'Mozilla/5.0'}
r=requests.get("https://www.amazon.com/",headers=kv)
print(r.status_code)
200
百度接口:
http://www.baidu.com/s?wd=keyword
360:
http://www.so.com/s?q=keyword
import requests
kv={'wd':'王泽坤'}
r=requests.get("http://www.baidu.com/s",params=kv)
print(r.text)
ip查询接口:
http://m.ip138.com/ip.asp?ip=ipaddress