错误提示:
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='baike.baidu.com', port=443):
Max retries exceeded with url: https://baike.baidu.com/item/%E5%88%98%E5%BE%B7%E5%8D%8E/114923
(Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fb51433af98>:
Failed to establish a new connection: [Errno -2] Name or service not known',))
或
requests.exceptions.ConnectionError: HTTPConnectionPool(host='www.sucaitianxia.net', port=80): Max retries exceeded with url: /e/search/result/index.p hp?page=418&searchid=14 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000000003F1E3C8>: Failed to establish a new con nection: [WinError 10060] 由于连接方在一段时间后没有正确答复或连接的主机没有反应,连接尝试失败。',))
经过一番查询,发现该错误是因为如下:
-
http的连接数超过最大限制,默认的情况下连接是Keep-alive的,所以这就导致了服务器保持了太多连接而不能再新建连接。
-
ip被封
-
程序请求速度过快。
解决办法如下:
第一种方法
try:
page1 = requests.get(ap)
except requests.exceptions.ConnectionError:
r.status_code = "Connection refused"
第二种方法:
request的连接数过多而导致Max retries exceeded
在header中不使用持久连接
'Connection': 'close'
或
requests.adapters.DEFAULT_RETRIES = 5
第三种方法:
针对请求请求速度过快导致程序报错。
解决方法可以参考以下例子:
import time
while 1:
try:
page = requests.get(url)
except:
print("Connection refused by the server..")
print("Let me sleep for 5 seconds")
print("ZZzzzz...")
time.sleep(5)
print("Was a nice sleep, now let me continue...")
continue
转自:http://www.chenxm.cc/article/255.html