python第三方库网址-python 第三方库requests连接url报错

本文记录了一次使用Python requests库抓取网页时遇到的问题,在Mac上正常运行的代码在Ubuntu服务器上出现连接被重置的错误。文章探讨了可能的原因,并提到在连续访问URL时更容易触发此错误。
摘要由CSDN通过智能技术生成

使用的语句是

page = requests.get( url , headers = self.header, timeout = 10 , verify = flag )

各变量的值分别为

url = "http://www.sbacn.org"

flag = False

self.header = {

"User-Agent" : "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:40.0) Gecko/20100101 Firefox/40.0",

"Accept" : "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",

"Accept-Language" : "zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3",

"Accept-Encoding" : "gzip, deflate",

}

报错内容为

Traceback (most recent call last):

File "bing.py", line 237, in

bing.titleGet(urls)

File "bing.py", line 195, in titleGet

page = self.dataRequest(url)

File "bing.py", line 86, in dataRequest

page = requests.get( url , headers = self.header, timeout = 10 , verify = flag )

File "/usr/lib/python2.7/site-packages/requests/api.py", line 69, in get

return request("get", url, params=params, **kwargs)

File "/usr/lib/python2.7/site-packages/requests/api.py", line 50, in request

response = session.request(method=method, url=url, **kwargs)

File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 468, in request

resp = self.send(prep, **send_kwargs)

File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 608, in send

r.content

File "/usr/lib/python2.7/site-packages/requests/models.py", line 734, in content

self._content = bytes().join(self.iter_content(CONTENT_CHUNK_SIZE)) or bytes()

File "/usr/lib/python2.7/site-packages/requests/models.py", line 657, in generate

for chunk in self.raw.stream(chunk_size, decode_content=True):

File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/response.py", line 326, in stream

data = self.read(amt=amt, decode_content=decode_content)

File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/response.py", line 282, in read

data = self._fp.read(amt)

File "/usr/lib64/python2.7/httplib.py", line 567, in read

s = self.fp.read(amt)

File "/usr/lib64/python2.7/socket.py", line 380, in read

data = self._sock.recv(left)

socket.error: [Errno 104] Connection reset by peer

我纳闷的是在我的mac上运行就没问题,但在服务器的ubuntu上运行就会报错,这是为什么?

而且我其实是抓了bing的搜索结果里10页的url,连续访问的时候就会报错,但我要是把这个url单独拿出来访问的时候就没问题.这是为什么?

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值