针对部分网站使用python请求出现403等,强制使用tls1.2发包的场景
如某盛优选小程序,fiddler重新发包正常,用程序请求失败
1.urllib版本
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
response = urllib.request.urlopen(url=urllib.request.Request(url=url, headers=headers),
data=data, context=context)
response = response.read().decode()
print(response)
2.requests版本
import ssl
import requests
from requests.adapters import HTTPAdapter, PoolManager
class MyAdapter(HTTPAdapter):
# 重写init_poolmanager方法
def init_poolmanager(self, connections, maxsize, block=False, **pool_kwargs):
self.poolmanager = PoolManager(num_pools=connections,
maxsize=maxsize,
block=block,
ssl_version=ssl.PROTOCOL_TLSv1_2)
s = requests.Session()
s.mount('https://', MyAdapter())
res = s.post(url, data=data, headers=headers)
text = res.text
3.在scrapy中强制使用TLS1.2的话需要在settings.py中设置以下代码
参考scrapy文档: https://docs.scrapy.org/en/latest/topics/settings.html#std-setting-DOWNLOADER_CLIENT_TLS_CIPHERS
DOWNLOADER_CLIENT_TLS_METHOD="TLSv1.2"
4.直接使用httpx请求
import httpx
res = httpx.post(url, data=data, headers=headers)
text = res.text